IEA Research for Education 
A Series of In-depth Analyses Based on Data of the International 
Association for the Evaluation of Educational Achievement (IEA) 


. | s ~_. ig ? 
a oust i .-s 
ce > ae 
: = 7° ier * 


Nathan Burroughs - Jacqueline Gardner - 
Youngjun Lee - Siwen Guo - Israel Touitou - 
Kimberly Jansen - William Schmidt 


Teaching for 
Excellence and 
moltiiay 


Analyzing Teacher Characteristics, Behaviors 
and Student Outcomes with TIMSS 


Y) Springer Open 


IEA Research for Education 


A Series of In-depth Analyses Based on Data 
of the International Association for the Evaluation 
of Educational Achievement (IEA) 


Volume 6 


Series Editors 


Seamus Hegarty, University of Warwick, UK, and Chair of IEA Publications and 
Editorial Committee 
Leslie Rutkowski, Indiana University, USA 


Editorial Board 


John Ainley, Australian Council for Educational Research, Australia 
Kadriye Ercikan, University of British Columbia, Canada 

Eckhard Klieme, German Institute for International Educational Research (DIPF), 
Germany 

Rainer Lehmann, Humboldt University of Berlin, Germany 

Fou-Lai Lin, National Taiwan Normal University, Chinese Taipei 
Marlaine Lockheed, Princeton University, USA 

Sarah Maughan, AlphaPlus Consultancy, UK 

Carina Omoeva, FHI 360, USA 

Elena Papanastasiou, University of Nicosia, Cyprus 

Valena White Plisko, Independent Consultant, USA 

Jonathan Plucker, John Hopkins University, USA 

Fernando Reimers, Harvard Graduate School of Education, USA 
David Rutkowski, Indiana University, USA 

Jouni Valijarvi, University of Jyvaskyla, Finland 

Hans Wagemaker, Senior Advisor to IEA, New Zealand 


The International Association for the Evaluation of Educational Achievement 
(IEA) is an independent nongovernmental nonprofit cooperative of national 
research institutions and governmental research agencies that originated in 
Hamburg, Germany in 1958. For over 60 years, IEA has developed and conducted 
high-quality, large-scale comparative studies in education to support countries’ 
efforts to engage in national strategies for educational monitoring and improvement. 

IEA continues to promote capacity building and knowledge sharing to foster 
innovation and quality in education, proudly uniting more than 60 member 
institutions, with studies conducted in more than 100 countries worldwide. 

IEA’s comprehensive data provide an unparalleled longitudinal resource for 
researchers, and this series of in-depth peer-reviewed thematic reports can be used 
to shed light on critical questions concerning educational policies and educational 
research. The goal is to encourage international dialogue focusing on policy matters 
and technical evaluation procedures. The resulting debate integrates powerful 
conceptual frameworks, comprehensive datasets and rigorous analysis, thus 
enhancing understanding of diverse education systems worldwide. 


More information about this series at http://www.springer.com/series/14293 


Nathan Burroughs - Jacqueline Gardner - 
Youngjun Lee - Siwen Guo - 

Israel Touitou - Kimberly Jansen - 
William Schmidt 


Teaching for Excellence 
and Equity 


Analyzing Teacher Characteristics, Behaviors 
and Student Outcomes with TIMSS 


ZF IEA o) Springer Open 


Nathan Burroughs 
CREATE for STEM 
Michigan State University 
East Lansing, MI, USA 


Youngjun Lee 
Michigan State University 
East Lansing, MI, USA 


Israel Touitou 
CREATE for STEM 
Michigan State University 


Jacqueline Gardner 
Office of K-12 Outreach 
Michigan State University 
East Lansing, MI, USA 


Siwen Guo 

College of Education 
Michigan State University 
East Lansing, MI, USA 


Kimberly Jansen 
Michigan State University 


East Lansing, MI, USA East Lansing, MI, USA 


William Schmidt 

Center for the Study of Curriculum Policy 
Michigan State University 

East Lansing, MI, USA 


ISSN 2366-1631 ISSN 2366-164X (electronic) 
IEA Research for Education 

ISBN 978-3-030-16150-7 ISBN 978-3-030-16151-4 (eBook) 
https://doi.org/10.1007/978-3-030-16151-4 


Library of Congress Control Number: 2019935481 


© International Association for the Evaluation of Educational Achievement (IEA) 2019. This book is an 
open access publication. 

Open Access This book is licensed under the terms of the Creative Commons Attribution- 
NonCommercial 4.0 International License (http://creativecommons.org/licenses/by-nc/4.0/), which per- 
mits any noncommercial use, sharing, adaptation, distribution and reproduction in any medium or format, 
as long as you give appropriate credit to the original author(s) and the source, provide a link to the 
Creative Commons license and indicate if changes were made. 

The images or other third party material in this book are included in the book’s Creative Commons 
license, unless indicated otherwise in a credit line to the material. If material is not included in the book’s 
Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the 
permitted use, you will need to obtain permission directly from the copyright holder. 

This work is subject to copyright. All commercial rights are reserved by the author(s), whether the whole 
or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, 
recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission 
or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar 
methodology now known or hereafter developed. Regarding these commercial rights a non-exclusive 
license has been granted to the publisher. 

The use of general descriptive names, registered names, trademarks, service marks, etc. in this publi- 
cation does not imply, even in the absence of a specific statement, that such names are exempt from the 
relevant protective laws and regulations and therefore free for general use. 

The publisher, the authors and the editors are safe to assume that the advice and information in this 
book are believed to be true and accurate at the date of publication. Neither the publisher nor the 
authors or the editors give a warranty, expressed or implied, with respect to the material contained 
herein or for any errors or omissions that may have been made. The publisher remains neutral with regard 
to jurisdictional claims in published maps and institutional affiliations. 


This Springer imprint is published by the registered company Springer Nature Switzerland AG 
The registered company address is: Gewerbestrasse 11, 6330 Cham, Switzerland 


Foreword 


IEA’s mission is to enhance knowledge about education systems worldwide, and to 
provide high-quality data that will support education reform and lead to better teaching 
and learning in schools. In pursuit of this aim, it conducts, and reports on, major studies 
of student achievement in literacy, mathematics, science, citizenship, and digital lit- 
eracy. These studies, most notably TIMSS, PIRLS, ICCS, and ICILS, are well estab- 
lished and have set the benchmark for international comparative studies in education. 

The studies have generated vast datasets encompassing student achievement, 
disaggregated in a variety of ways, along with a wealth of contextual information 
which contains considerable explanatory power. The numerous reports that have 
emerged from them are a valuable contribution to the corpus of educational research. 

Valuable though these detailed reports are, IEA’s goal of supporting education 
reform needs something more: deep understanding of education systems and the many 
factors that bear on student learning advances through in-depth analysis of the global 
datasets. IEA has long championed such analysis and facilitates scholars and policy- 
makers in conducting secondary analysis of our datasets. So, we provide software such 
as the International Database Analyzer to encourage the analysis of our datasets, 
support numerous publications including a peer-reviewed journal—Large-scale 
Assessment in Education—dedicated to the science of large-scale assessment and 
publishing articles that draw on large-scale assessment databases, and organize a 
biennial international research conference to nurture exchanges between researchers 
working with IEA data. 

The IEA Research for Education series represents a further effort by IEA to 
capitalize on our unique datasets, so as to provide powerful information for 
policymakers and researchers. Each report focuses on a specific topic and is 
produced by a dedicated team of leading scholars on the theme in question. Teams 
are selected on the basis of an open call for tenders; there are two such calls a year. 
Tenders are subject to a thorough review process, as are the reports produced 
(Full details are available on the IEA website). 

This sixth volume in the series is concerned with teacher quality and educational 
outcomes. Regarded as the most vital school resource, teachers are key to under- 
standing the “black box” of education. To that end, the current volume examines the 
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link between teacher effectiveness and student outcomes from several different per- 
spectives. Especially important in this research is understanding whether better 
teachers can mitigate the known deleterious effect of low socioeconomic status. 
Literature on the links between social class and achievement abounds; however, 
questions remain concerning the role of the teacher and teaching quality in this 
relationship. 

To begin a study on the importance of teacher effectiveness, the authors draw on 
data collected over 20 years by the IEA’s Trends in International Mathematics and 
Science Study (TIMSS). Although student outcomes, namely, achievement, figure 
prominently in this work, the authors use a wide array of contextual information to 
bring their narrative to life. Inputs, such as teacher experience, teachers’ perceptions 
of their own levels of preparedness, and the degree to which the implemented cur- 
riculum is aligned to the educational system’s curricular expectations, are modeled as 
explanations for system-level variation in achievement. The authors further examine 
whether there are stable patterns over time, educational system, and grade in the 
investigated relationships. As is the bane of much social science research, particularly 
the cross-cultural sort, revealing consistent patterns across dozens of heterogeneous 
countries over time is no mean feat. Rather, the authors found substantial variation in 
teacher characteristics, behavior, and perceptions, and in the way in which these 
variables relate to mathematics achievement. Here, the old adage about change being 
the only constant is appropriate. 

In spite of an absence of conclusive patterns, this book uses high-quality data to 
underscore an axiom of international research in education: the local context matters. To 
ignore this important result is to risk wasting time and effort at the expense of children. 
As the authors wisely note, “Simple transference of policy ideas that have enjoyed 
apparent success in one educational system can yield unexpected (or even disastrous) 
consequences in another” (Chap. 8). A key takeaway from this book is that international 
assessment databases, such as TIMSS, are rich resources for testing these sorts of the- 
ories and for understanding whether and when a successful policy initiative in one 
country can be expected to succeed in other contexts. A second conclusion worth 
repeating here is that, based on extensive analyses, teachers with similar backgrounds 
and experiences can be expected to produce different results in the classroom. Again, the 
local context matters. When teachers are assigned an undue share of responsibility, this 
shifts emphasis away from society at large and the role that we all play in ensuring 
high-quality education for the next generation. 

Future publications in this series will include a novel exploration of student 
profiles using variables related to motivation, affect, and attitudes about school 
mathematics, and an in-depth investigation into the nature and extent of students’ 
misconceptions and misunderstandings related to core concepts in mathematics and 
physics across grades four, eight, and 12. 


Seamus Hegarty 
Leslie Rutkowski 
Series editors 
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Chapter 1 ®) 
The Role of International Assessments ly 
in Understanding Teacher Quality 


Abstract Internationally, high-quality teachers are often considered to play an 
essential role in good student outcomes, but there is little research on the relationship 
between measures of teacher effectiveness and student achievement. Variation 
across countries in how diverse measures of teacher quality are related to student 
outcomes has made cross-country transfer of educational ideas difficult, since, 
without understanding if or why policies work in different national contexts, 
policymakers and researchers have no solid basis for devising the best strategies 
for improving teacher effectiveness. IEA’s Trends in International Mathematics and 
Science Study (TIMSS) collects extensive information about teacher background and 
practices across countries, and therefore can potentially be used to address important 
questions about the role of teachers in influencing student outcomes. In this study, 
analysis of the TIMSS data prompts a series of research questions, designed to 
investigate whether there are identifiable international trends in teacher quality and 
instructional metrics over time while focusing on issues of educational equity, and 
examine the association between teacher characteristics and behaviors and mean 
student outcomes. The chapter concludes with a brief outline of the remainder of the 
report. 


Keywords Educational equity - Student achievement - Teacher quality - Trends in 
International Mathematics and Science Study (TIMSS) 


1.1. Introduction 


While it has become commonplace to argue that high-quality teachers are essential 
to student learning, unfortunately, there is little clarity about the best means for 
improving teacher effectiveness, with major consequences for policymakers and 
researchers alike. Over the last several decades, education reformers have attempted 
to improve the capacity of the teacher labor force and the quality of instructional 
content, spawning a voluminous research literature in the process. However, the 
relationships between measures of teacher effectiveness and student outcomes, 
whether understood as mean achievement or equity, are inconsistent, and this, in 
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turn, has raised serious questions about the best approach for achieving policy goals. 
In this sense, the ambiguity in the research literature has left policymakers with little 
direction as to the best approaches to reform. In addition, variation across countries 
in how various measures of teacher quality are related to student outcomes has made 
cross-country transfer of educational ideas difficult, since without understanding 
whether (or why) policies work differently in different national contexts, it is hard 
to know whether (or when) a particular policy should be adopted (Plucker 2014). 
The central aim of this report was to investigate what international comparative 
assessments can reveal about the role of teachers in influencing student outcomes. 
While the bulk of existing research from the United States suggests that teacher 
credentials have a limited impact on student achievement, it remains unclear whether 
this finding is more broadly applicable. If the limited impact of teacher credentials in 
the United States is due to the specifics of the country’s teacher preparation system 
(or educational system more broadly) or statistical issues (for example, more limited 
variation in how teachers are educated), research would suggest that the United 
States should seek solutions in other countries. However, if the (weak) relationship 
holds across educational systems, then policymakers and researchers will have to 
determine whether formal teacher preparation is an effective lever for improving 
student outcomes. 


1.2 Linking Teaching Quality to Student Outcomes 


Ever since its foundation in the late 1950s, the International Association for the 
Evaluation of Educational Achievement (IEA) has focused on providing high-quality 
impartial comparative data on student learning and contexts, aimed at enabling 
researchers and policymakers to unpack the “black box” of classroom instruction 
(Schmidt et al. 2018). Beginning with the original pilot study of 12 countries in the 
late 1960s, and continuing with the First International Mathematics Study (FIMS), 
the Second International Mathematics Study (SIMS), and the Trends in International 
Mathematics and Science Study (TIMSS), IEA sponsored studies have attempted 
to identify the key mechanisms of classroom instruction. Since 1995, successive 
cycles of TIMSS have collected extensive information about teacher background 
and practices across countries; such data can therefore potentially be used to address 
important questions about the role of teachers in influencing student outcomes, and 
is ideally suited to examining the relationship between teacher quality and student 
outcomes. 

The design of TIMSS renders it feasible to focus on the role of teachers. There are 
two major long-term large-scale international assessments of student mathematics: 
IEA’s TIMSS, and the Programme of International Student Assessment (PISA) run 
by the Organisation for Economic Cooperation and Development (OECD). Both 
TIMSS and PISA randomly select a group of representative schools, but whereas the 


1.2 Linking Teaching Quality to Student Outcomes 3 


PISA study selects a group of 15-year-old students from within those schools without 
linking them to specific teachers, TIMSS selects intact classrooms and collects 
extensive information about teacher background and practice. As a consequence, 
TIMSS data provides a unique opportunity to examine the impact of teachers on a 
representative sample of students in multiple countries across time. As TIMSS also 
tests students at two different grades (namely grades four and eight), it is also possible 
to examine the specific impacts of teacher characteristics and teaching practices on 
students at two different stages of their learning. 


1.3. Conceptual Framework and Research Questions 


In this study, we took advantage of the design of TIMSS to examine key teacher 
characteristics (experience, education, and preparedness to teach) and teacher 
behaviors (instructional time and instructional content) and assessed how these were 
related to student outcomes using data collected for both grade four and grade eight 
during the multiple past cycles of TIMSS. We used national curriculum data collected 
by the TIMSS assessments to assess the relationship between teacher instruction 
and national curricular standards (e.g., instructional alignment) and educational 
outcomes. We also focused on the distributional impact of curriculum and instruction 
on students, with attention paid to overall and socioeconomic status based inequality 
at the student and country level. We explored the evolution of these associations 
over time using multiple methods, including regression, fixed effect, and structural 
equation modeling. We also paid close attention to the methodological complexities 
involved with using TIMSS data and their impact in examining the relationships 
between teacher measures and student outcomes. 

Our primary focus was student learning in mathematics. Although international 
assessments have also examined reading and science achievement, the research 
literature on the role of instructional content is much better grounded for mathematics, 
and hence presents a better example of the impact of schools and teachers on 
student outcomes because learning of mathematics takes place principally inside the 
classroom. Whereas students may be well exposed to reading, the use of language, 
or basic scientific concepts outside school, they tend to have much more limited 
exposure to mathematical concepts before they enter school (Sparks 2017). 

We examine teacher effectiveness along multiple dimensions, including both 
traditional measures (teacher experience and teacher education) and more 
infrequently employed indicators (instructional time and content, and preparedness to 
teach). Although others have examined the contribution of these elements to student 
achievement, to our knowledge, there has been no previous research into the joint 
effects of these factors using a multi-year, multi-country model. The research outlined 
in this report is therefore novel in assessing the robustness of these relationships 
across countries treating each iteration of the TIMSS as a separate sample, thereby 
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testing the replicability and reproducibility of analyses based on only one year of 
data. Our approach also enabled us to examine whether the significant cross-country 
variation in some of the observed associations (especially the relationships of 
teacher experience and teacher education with achievement) was more consistent 
within countries. Examining trends provides a means to evaluate the success of 
an education system in strengthening teacher quality, improving coherence, and 
reducing inequality. This report builds on an initial valuable exploration of these 
topics undertaken by Mullis et al. (2016), in which TIMSS trend data was used 
to demonstrate considerable national-level curriculum reform (leading to greater 
instructional coherence), strengthened teacher preparation requirements, a reduction 
in standard deviations in student performance, but little change in the amount of time 
devoted to teacher professional development or to mathematics instruction. 

The conceptual model that we used in this report builds on the work of 
Blomeke et al. (2016), who applied a structural equation modeling approach 
to the TIMSS 2011 data, analyzing each country separately. Their model 
included teacher observable characteristics (years of experience, college major, 
and specialization), professional development (participation in broad mathematics 
instruction professional development, specific mathematics instruction professional 
development, and collaborative professional development), and teacher preparedness 
(using preparedness to teach numbers, geometry, and data indices) as direct predictors 
of student achievement. The relationship of these predictors with student outcomes 
are mediated by instructional quality, operationalized as a latent variable derived 
from clarity of instruction, supportive climate, and cognitive activation indicators. 
Blomeke et al. (2016) also controlled for student gender and books in the home in 
their analysis. 

While our model is based on the Blomeke model, we incorporated several 
additional components. Blomeke et al. (2016) noted a weak relationship between 
instructional quality and student outcomes. As it stands, the Blomeke model 
addresses the “how” without the “what;” the weak relationships they observed may 
be explained by the absence of measures of instructional content (what teachers are 
teaching) and of how long they spent on this task. Given the strong research base 
on the effects of opportunity to learn and instructional time on student learning 
(see Chap. 2), we believed that including such factors as additional mediating 
variables would influence the estimates of instructional quality and greatly strengthen 
the model as a whole. We thus also explored the relationship between teacher 
characteristics, and content coverage and instructional time, which despite obvious 
plausibility has received little attention in the research literature. 

Bearing all these considerations in mind, we developed four key questions to 
guide our research: 


(1) Are there identifiable trends in teacher quality and instructional metrics over 
time? 

(2) What are the relationships between student achievement and different types of 
teacher quality and instructional metrics? 
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(3) How stable are these relationships across time and statistical method? 
(4) What are the relationships between student equity and teacher quality and 
instructional metrics? 


In this report, we aim to address each of these questions in turn. We begin by 
reviewing the existing research literature on teacher effectiveness in Chap. 2. Next, 
in Chap. 3, we focus on the variables we used for our analyses and consider some of 
the methodological issues involved. Turning to our research questions, in Chap. 4, we 
present descriptive statistics for teacher quality and instructional metrics, establishing 
changes over time and potential trends in the education-system-level means reported 
in the TIMSS data. In Chap. 5, we further analyze multiple cycles of the data using 
ordinary least squares regression, multilevel, and fixed-effect models, while paying 
close attention to the stability of estimates across time and according to each of 
the different statistical methods, concentrating on the relationship between teacher 
quality variables and student outcomes. Expanding on this analysis, in Chap. 6 we 
present a multilevel structural equation model using TIMSS 2011 data that extends 
the earlier work of Blomeke et al. (2016). In Chap. 7, we shift our focus to our final 
research question regarding issues of educational equity, examining the associations 
between teacher characteristics and behaviors and differences among students, rather 
than average outcomes. Finally, we conclude in Chap. 8 with a discussion of the 
overall research findings, implications for policymakers and researchers, a review of 
potential limitations of the work, and suggestions for future research. 
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Chapter 2 ®) 
A Review of the Literature on Teacher Ghosk for 
Effectiveness and Student Outcomes 


Abstract Researchers agree that teachers are one of the most important 
school-based resources in determining students’ future academic success and lifetime 
outcomes, yet have simultaneously had difficulties in defining what teacher 
characteristics make for an effective teacher. This chapter reviews the large body 
of literature on measures of teacher effectiveness, underscoring the diversity of 
methods by which the general construct of “teacher quality” has been explored, 
including experience, professional knowledge, and opportunity to learn. Each 
of these concepts comprises a number of different dimensions and methods of 
operationalizing. Single-country research (and particularly research from the United 
States) is distinguished from genuinely comparative work. Despite a voluminous 
research literature on the question of teacher quality, evidence for the impact 
of teacher characteristics (experience and professional knowledge) on student 
outcomes remains quite limited. There is a smaller, but more robust set of findings 
for the effect of teacher support on opportunity to learn. Five measures may 
be associated with higher student achievement: teacher experience (measured by 
years of teaching), teacher professional knowledge (measured by education and 
self-reported preparation to teach mathematics), and teacher provision of opportunity 
to learn (measured by time on mathematics and content coverage). These factors 
provide the basis for a comparative cross-country model. 


Keywords Opportunity to learn - Teacher education - Teacher experience - 
Teacher quality - Trends in International Mathematics and Science Study (TIMSS) 


2.1 Defining Teacher Effectiveness 


Researchers agree that teachers are one of the most important school-based resources 
in determining students’ future academic success and lifetime outcomes (Chetty 
et al. 2014; Rivkin et al. 2005; Rockoff 2004). As a consequence, there has been 
a strong emphasis on improving teacher effectiveness as a means to enhancing 
student learning. Goe (2007), among others, defined teacher effectiveness in terms of 
growth in student learning, typically measured by student standardized assessment 
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results. Chetty et al. (2014) found that students taught by highly effective teachers, as 
defined by the student growth percentile (SGPs) and value-added measures (VAMs), 
were more likely to attend college, earn more, live in higher-income neighborhoods, 
save more money for retirement, and were less likely to have children during their 
teenage years. This potential of a highly effective teacher to significantly enhance the 
lives of their students makes it essential that researchers and policymakers properly 
understand the factors that contribute to a teacher’s effectiveness. However, as we will 
discuss in more detail later in this report, studies have found mixed results regarding 
the relationships between specific teacher characteristics and student achievement 
(Wayne and Youngs 2003). In this chapter, we explore these findings, focusing on the 
three main categories of teacher effectiveness identified and examined in the research 
literature: namely, teacher experience, teacher knowledge, and teacher behavior. Here 
we emphasize that much of the existing body of research is based on studies from 
the United States, and so the applicability of such national research to other contexts 
remains open to discussion. 


2.2 Teacher Experience 


Teacher experience refers to the number of years that a teacher has worked as 
a classroom teacher. Many studies show a positive relationship between teacher 
experiences and student achievement (Wayne and Youngs 2003). For example, using 
data from 4000 teachers in North Carolina, researchers found that teacher experience 
was positively related to student achievement in both reading and mathematics 
(Clotfelter et al. 2006). Rice (2003) found that the relationship between teacher 
experience and student achievement was most pronounced for students at the 
secondary level. Additional work in schools in the United States by Wiswall (2013), 
Papay and Kraft (2015), and Ladd and Sorenson (2017), and a Dutch twin study by 
Gerritsen et al. (2014), also indicated that teacher experience had a cumulative effect 
on student outcomes. 

Meanwhile, other studies have failed to identify consistent and statistically 
significant associations between student achievement and teacher experience 
(Blomeke et al. 2016; Gustaffsson and Nilson 2016; Hanushek and Luque 2003; 
Luschei and Chudgar 2011; Wilson and Floden 2003). Some research from the 
United States has indicated that experience matters very much early on in a teacher’s 
career, but that, in later years, there were little to no additional gains (Boyd et al. 
2006; Rivkin et al. 2005; Staiger and Rockoff 2010). In the first few years of a 
teacher’s career, accruing more years of experience seems to be more strongly related 
to student achievement (Rice 2003). Rockoff (2004) found that, when comparing 
teacher effectiveness (understood as value-added) to student test scores in reading 
and mathematics, teacher experience was positively related to student mathematics 
achievement; however, such positive relationships leveled off after teachers had 
gained two years of teaching experience. Drawing on data collected from teachers of 
grades four to eight between 2000 and 2008 within a large urban school district in the 
United States, Papay and Kraft (2015) confirmed previous research on the benefits 
experience can add to a novice teacher’s career. They found that student outcomes 
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increased most rapidly during their teachers’ first few years of employment. They 
also found some further student gains due to additional years of teaching experience 
beyond the first five years. The research of Pil and Leana (2009) adds additional 
nuance; they found that acquiring teacher experience at the same grade level over a 
number of years, not just teacher experience in general (i.e. at multiple grades), was 
positively related to student achievement. 


2.3 Teacher Professional Knowledge 


A teacher’s professional knowledge refers to their subject-matter knowledge, 
curricular knowledge, and pedagogical knowledge (Collinson 1999). This 
professional knowledge is influenced by the undergraduate degrees earned by a 
teacher, the college attended, graduate studies undertaken, and opportunities to 
engage with on-the job training, commonly referred to as professional development 
(Collinson 1999; Rice 2003; Wayne and Youngs 2003). After undertaking in-depth 
quantitative analyses of the United States’ 1993-1994 Schools and Staffing 
Survey (SASS) and National Assessment of Educational Progress (NAEP) data 
sets, Darling-Hammond (2000) argued that measures of teacher preparation and 
certification were by far the strongest correlates of student achievement in reading 
and mathematics, after controlling for student poverty levels and language status. 
As with experience, research on the impact of teacher advanced degrees, 
subject specializations, and certification has been inconclusive, with several 
studies (Aaronson et al. 2007; Blomeke et al. 2016; Hanushek and Luque 
2003; Harris and Sass 2011; Luschei and Chudgar 2011) suggesting weak, 
inconsistent, or non-significant relationships with student achievement. However, 
several international studies comparing country means found that teacher degrees 
(Akiba et al. 2007; Gustaffsson and Nilson 2016; Montt 2011) were related to student 
outcomes, as did Woessman’s (2003) student-level study of multiple countries. 


2.3.1 Undergraduate Education 


In their meta-analysis of teacher effectiveness, Wayne and Youngs (2003) found 
three studies that showed some relationship between the quality of the undergraduate 
institution that a teacher attended and their future students’ success in standardized 
tests. In a thorough review of the research on teacher effectiveness attributes, 
Rice (2003) found that the selectivity of undergraduate institution and the teacher 
preparation program may be related to student achievement for students at the high 
school level and for high-poverty students. 

In terms of teacher preparation programs, Boyd et al. (2009) found that overall 
these programs varied in their effectiveness. In their study of 31 teacher preparation 
programs designed to prepare teachers for the New York City School District, Boyd 
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et al. (2009) drew from data based on document analyses, interviews, surveys of 
teacher preparation instructors, surveys of participants and graduates, and student 
value-added scores. They found that if a program was effective in preparing teachers 
to teach one subject, it tended to also have success in preparing teachers to teach other 
subjects as well. They also found that teacher preparation programs that focused on 
the practice of teaching and the classroom, and provided opportunities for teachers 
to study classroom practices, tended to prepare more effective teachers. Finally, they 
found that programs that included some sort of final project element (such as a 
personal research paper, or portfolio presentation) tended to prepare more effective 
teachers. 

Beyond the institution a teacher attends, the coursework they choose to take 
within that program may also be related to their future students’ achievement. These 
associations vary by subject matter. A study by Rice (2003) indicated that, for 
teachers teaching at the secondary level, subject-specific coursework had a greater 
impact on their future students’ achievement. Similarly Goe (2007) found that, for 
mathematics, an increase in the amount of coursework undertaken by a trainee 
teacher was positively related to their future students’ achievement. By contrast, 
the meta-analysis completed by Wayne and Youngs (2003) found that, for history 
and English teachers, there was no evidence of a relationship between a teacher’s 
undergraduate coursework and their future students’ achievement in those subjects. 


2.3.2 Graduate Education 


In areview of 14 studies, Wilson and Floden (2003) were unable to identify consistent 
relationships between a teacher’s level of education and their students’ achievement. 
Similarly, in their review of data from 4000 teachers in North Carolina, Clotfelter et al. 
(2006) found that teachers who held a master’s degree were associated with lower 
student achievement. However, specifically in terms of mathematics instruction, 
teachers with higher degrees and who undertook more coursework during their 
education seem to be positively related to their students’ mathematics achievement 
(Goe 2007). Likewise, Harris and Sass (2011) found that there was a positive 
relationship between teachers who had obtained an advanced degree during their 
teaching career and their students’ achievement in middle school mathematics. 
They did not find any significant relationships between advanced degrees and 
student achievement in any other subject area. Further, using data from the United 
States’ Early Childhood Longitudinal Study (ECLS-K), Phillips (2010) found that 
subject-specific graduate degrees in elementary or early-childhood education were 
positively related to students’ reading achievement gains. 
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2.3.3 Certification Status 


Another possible indicator of teacher effectiveness could be whether or not a teacher 
holds a teaching certificate. Much of this research has focused on the United 
States, which uses a variety of certification approaches, with lower grades usually 
having multi-subject general certifications and higher grades requiring certification 
in specific subjects. Wayne and Youngs (2003) found no clear relationship between 
US teachers’ certification status and their students’ achievement, with the exception 
of the subject area of mathematics, where students tended have higher test scores 
when their teachers had a standard mathematics certification. Rice (2003) also found 
that US teacher certification was related to high school mathematics achievement, 
and also found that there was some evidence of a relationship between certification 
status and student achievement in lower grades. Meanwhile, in their study of grade 
one students, Palardy and Rumberger (2008) also found evidence that students made 
greater gains in reading ability when taught by fully certified teachers. 

In a longitudinal study using data from teachers teaching grades four and five 
and their students in the Houston School District in Texas, Darling-Hammond 
et al. (2005) found that those teachers who had completed training that resulted 
in a recognized teaching certificate were more effective that those who had no 
dedicated teaching qualifications. The study results suggested that teachers without 
recognized US certification or with non-standard certifications generally had negative 
effects on student achievement after controlling for student characteristics and prior 
achievement, as well as the teacher’s experience and degrees. The effects of teacher 
certification on student achievement were generally much stronger than the effects 
for teacher experience. Conversely, analyzing data from the ECLS-K, Phillips (2010) 
found that grade one students tended to have lower mathematics achievement gains 
when they had teachers with standard certification. In sum, the literature the influence 
of teacher certification remains deeply ambiguous. 


2.3.4 Professional Development 


Although work by Desimone et al. (2002, 2013) suggested that professional 
development may influence the quality of instruction, most researchers found that 
teachers’ professional development experiences showed only limited associations 
with their effectiveness, although middle- and high-school mathematics teachers 
who undertook more content-focused training may be the exception (Blomeke et al. 
2016; Harris and Sass 2011). In their meta-analysis of the effects of professional 
development on student achievement, Blank and De Las Alas (2009) found that 
16 studies reported significant and positive relationships between professional 
development and student achievement. For mathematics, the average effect size of 
studies using a pre-post assessment design was 0.21 standard deviations. 
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Analyzing the data from six data sets, two from the Beginning Teacher Preparation 
Survey conducted in Connecticut and Tennessee, and four from the United States 
National Center for Education Statistics’ National Assessment of Educational 
Progress (NAEP), Wallace (2009) used structural equation modeling to find that 
professional development had a very small, but occasionally statistically significant 
effect on student achievement. She found, for example, that for NAEP mathematics 
data from the year 2000, 1.2 additional hours of professional development per year 
were related to an increase in average student scores of 0.62 points, and for reading, 
an additional 1.1 h of professional development were related to an average increase 
in student scores of 0.24 points. Overall, Wallace (2009) identified professional 
development had moderate effects on teacher practice and some small effects on 
student achievement when mediated by teacher practice. 


2.3.5 Teacher Content Knowledge 


Of course, characteristics like experience and education may be imperfect proxies 
for teacher content knowledge; unfortunately, content knowledge is difficult to assess 
directly. However, there is a growing body of work suggesting that teacher content 
knowledge may associated with student learning. It should be noted that there is an 
important distinction between general content knowledge about a subject (CK) and 
pedagogical content knowledge (PCK) specifically related to teaching that subject, 
each of which may be independently related to student outcomes (Baumert et al. 
2010). 

Studies from the United States (see for example, Chingos and Peterson 2011; 
Clotfelter et al. 2006; Constantine et al. 2009; Hill et al. 2005; Shuls and Trivitt 2015) 
have found some evidence that higher teacher cognitive skills in mathematics are 
associated with higher student scores. Positive associations between teacher content 
knowledge and student outcomes were also found in studies based in Germany 
(Baumert et al. 2010) and Peru (Metzler and Woessman 2012), and in a comparative 
study using Programme for the International Assessment of Adult Competencies 
(PIAAC) data undertaken by Hanushek et al. (2018). These findings are not universal, 
however, other studies from the United States (Blazar 2015; Garet et al. 2016; Rockoff 
etal. 2011) failed to find a statistically significant association between teacher content 
knowledge and student learning. 

The studies we have discussed all used some direct measure of teacher 
content knowledge. An alternative method of assessing mathematics teacher content 
knowledge is self-reported teacher preparation to teach mathematics topics. Both 
TIMSS and IEA’s Teacher Education and Development Study in Mathematics 
(TEDS-M, conducted in 2007-2008) have included many questions, asking teachers 
to report on their preparedness to teach particular topics. Although Luschei and 
Chudgar (2011) and Gustafsson and Nilson (2016) found that these items had a 
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weak direct relationship to student achievement across countries, other studies have 
suggested that readiness is related to instructional quality (Blomeke et al. 2016), as 
well as content knowledge and content preparation (Schmidt et al. 2017), suggesting 
that instructional quality may have an indirect effect on student learning. 


2.4 Teacher Behaviors and Opportunity to Learn 


Although the impact of teacher characteristics (experience, education, and 
preparedness to teach) on student outcomes remains an open question, there is 
much a much more consistent relationship between student achievement and teacher 
behaviors (instructional time and instructional content), especially behaviors related 
instructional content. Analyzing TIMSS, Schmidt et al. (2001) found an association 
between classroom opportunity to learn (OTL), interpreted narrowly as student 
exposure to instructional content, and student achievement. In a later study using 
student-level PISA data, Schmidt et al. (2015) identified a robust relationship 
between OTL and mathematics literacy across 62 different educational systems. The 
importance of instructional content has been recognized by national policymakers, 
and has helped motivate standards-based reform in an effort to improve student 
achievement, such as the Common Core in the United States (Common Core 
Standards Initiative 2018). However, we found that there was little research on 
whether teacher instructional content that aligned with national standards had 
improved student learning; the only study that we were able to identify found that 
such alignment had only very weak associations with student mathematics scores 
(Polikoff and Porter 2014). Student-reported data indicates that instructional time 
(understood as classroom time on a particular subject) does seem to be related to 
mathematics achievement (Cattaneo et al. 2016; Jerrim et al. 2017; Lavy 2015; Rivkin 
and Schiman 2015; Woessman 2003). 


2.5 Conclusion 


This review of the literature simply brushes the surface of the exceptional 
body of work on the relationship between student achievement and teacher 
characteristics and behaviors. Whether analyzing US-based, international, or the 
(limited) number of comparative studies, the associations between easily measurable 
teacher characteristics, like experience and education, and student outcomes in 
mathematics, remains debatable. In contrast, there is more evidence to support the 
impact of teacher behaviors, such as instructional content and time on task, on student 
achievement. Our goal was to incorporate all these factors into a comparative model 
across countries, with the aim of determining what an international cross-national 
study like TIMSS could reveal about the influence of teachers on student outcomes 
in mathematics. The analysis that follows draws on the existing body of literature 
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on teacher effectiveness, which identified key teacher factors that may be associated 
with higher student achievement: teacher experience, teacher professional knowledge 
(measured by education and self-reported preparation to teach mathematics), and 
teacher provision of opportunity to learn (time on mathematics and content coverage). 
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Chapter 3 M®) 
Teacher Variables, and Modeling si 
the Relationships Between Teacher 
Characteristics, Teacher Behaviors, 

and Student Outcomes 


Abstract After a brief discussion of the history of IEA’s Trends in International 
Mathematics and Science Study (TIMSS), and the implications this has had for 
comparative research into the influence of teachers on student outcomes, this chapter 
details the key variables that are employed in the remainder of the book. These include 
several student-level control variables (such as gender, socioeconomic status, and 
language spoken in the home), teacher-reported measures (experience, education, 
gender, and time spent on teaching mathematics), and newly developed instruments 
based on TIMSS responses related to teacher self-efficacy and instructional 
alignment. All of the analyses in this report make use of TIMSS data, merging several 
distinct instruments: student assessments, teacher background surveys, and national 
background surveys. The design of TIMSS has a number of important implications. 
Relationships between variables should be treated as associations rather than strictly 
causal, thus a multi-model analytical strategy is needed to test the robustness of 
statistical results and improve confidence in the reliability of analyses. Relationships 
were examined across time, across different aggregations of data, and using different 
statistical procedures, and are discussed in more detail in later chapters. Many features 
of TIMSS and the topics included in the survey have changed considerably over time 
and some key variables of interest were not included in earlier versions of the TIMSS, 
greatly restricting the available sample when analyzing country-level trends (whether 
as means or regression coefficients). The complex sampling design has important 
implications for statistical modeling and analysis, and for purposes of analysis, the 
models in subsequent chapters ignore school effects. Multilevel and classroom-mean 
models treat each classroom as existing independently, rather than being clustered 
within schools. Given the acknowledged impact of schools, and the importance of 
within-school, between-classroom heterogeneity, any conclusions drawn from the 
analysis should be treated with caution. 


Keywords Opportunity to learn - Teacher education - Teacher experience - 
Teacher quality - Trends in International Mathematics and Science Study (TIMSS) 
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3.1 The TIMSS Dataset 


All of the analyses in this report make use of IEA’s Trends in International 
Mathematics and Science Study (TIMSS) data, merging several distinct instruments: 
student assessments, teacher background surveys, and national background surveys. 
The design of TIMSS uses a stratified random sampling to select a representative 
sample of schools at each grade level of the study, and then to sample intact 
classrooms within those schools. Separate surveys are conducted at grades four and 
eight. Since the first cycle of TIMSS in 1995, there have been four subsequent 
iterations at both grades four and eight (in 2003, 2007, 2011, 2015), as well as an 
additional survey at grade eight undertaken in 1999. 

The design of TIMSS has a number of important implications for researchers. 
First, despite continuing over time, the TIMSS is not a true longitudinal sample, 
but rather a series of cross-sectional studies. Because TIMSS does not sample the 
same group of students, teachers, or schools across time, many of the most common 
identification strategies employed by researchers to account for unobserved variables 
and other biases are not available. All of the relationships between variables should 
therefore be treated as suggested associations rather than strictly causal. To cope with 
this limitation, we adopted a multi-model analytical strategy to test for the robustness 
of statistical results. The aim was to improve confidence in the reliability of analyses 
by examining the stability of relationships across time, across different aggregations 
of data, and using different statistical procedures. These approaches are discussed in 
more detail in later chapters. 

Second, many features of TIMSS have changed considerably over time. Some 
of the key variables of interest, such as teacher preparedness to teach mathematics 
topics, or the mathematics topics expected to be taught according to the national 
research coordinators, were not included in earlier cycles of TIMSS. The list of 
topics included in TIMSS has also changed considerably since 1995. For example, 
although the literature suggests that receipt of professional development could be 
related to student outcomes (see for example Blank and De Las Alas 2009), the 
variation in how this construct is incorporated in different cycles of TIMSS is too 
diverse (in our judgment) to permit its consideration as a variable in multi-year 
statistical models.! Similarly, participating educational systems differ from one cycle 
of TIMSS to another; they may be present in one cycle of TIMSS but not in another, 
or may only participate at grade four but not at grade eight. Such variations greatly 
restrict the available sample when analyzing education-system-level trends (whether 
means or regression coefficients). For example, Germany did not take part at grade 
four in 2003, and only participated at grade eight in 1995. 

Thirdly, the complex sampling design of TIMSS has important implications 
for statistical modeling and analysis. Because typically only one intact classroom 
is selected in a given school, it is very difficult to distinguish classroom- from 


'For example, teachers were asked whether they had received professional development related to 
critical thinking in 2003 and 2007, but not in 2011 and 2015. By contrast, they were asked about 
professional development focused on student needs in 2011 and 2015, but not 2003 and 2007. 
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school-level effects. Therefore, for analytical purposes, we ignored school-level 
effects in our models. Multilevel and classroom-mean models treat each classroom 
as existing independently, rather than being clustered within schools. Given 
the acknowledged impact of schools, and the importance of within-school 
between-classroom heterogeneity, this amounts to an important qualification to 
any conclusions drawn from our analysis (Schmidt et al. 2015). Readers should 
understand that TIMSS samples a random population of classrooms meant to reflect 
an entire education system’s population, rather than a straightforward random sample 
of all students or teachers. As such, strictly speaking, the associations explored in 
this study should be interpreted as reflecting representative classrooms, rather than 
all of an educational system’s teachers and students. 

The complex sampling design, which stratifies within blocks of schools, giving 
different weights to different schools (and hence classrooms), means that standard 
calculation of standard errors (and hence statistical significance) is inadvisable. 
Instead, TIMSS requires a jackknifing procedure, which produces more accurate 
standard error estimates but at the cost of much greater computational burdens 
(especially for multilevel models). It also introduces problems with subdividing types 
of classrooms, as we discuss in Chap. 7. 

The educational systems examined in our analyses are usually referred to as 
“countries.” This is for ease of reading, but it should be noted that there are a number 
of systems that are not countries as such, but are units with a degree of educational 
autonomy that have participated in TIMSS following the same standards for sampling 
and testing. 

A final word of caution about the analyses in forthcoming chapters. Cross-country 
comparisons of teacher characteristics and behaviors inevitably raise questions about 
the consistency of these concepts across different cultural contexts. This is a particular 
problem for survey results, since along with difficulties in translation, there may be 
problems related to culturally-specific interpretations; these are some of the inevitable 
challenges encountered when conducting comparative research. 


3.2 Operationalization of Variables 


The variables in this study are drawn from the TIMSS database (see https://www. 
iea.nl/data). Additional details can be found in the TIMSS 2015 user guide (Foy 
2017) and other TIMSS user guides and technical reports (https://timssandpirls.bce. 
edu/isc/publications.html). The first three variables that we included in the various 
statistical models are student control variables that have been associated with student 
outcomes and/or different learning opportunities, which means that excluding them 
could result in spurious relationships between teacher quality and student outcomes. 
These are: student gender, number of books in the home (a common proxy measure 
of socioeconomic status reflecting both parental education and parental income), and 
whether the student speaks the language of the test at home (Chudgar and Luschei 
2009). The remaining variables are teacher variables that the research literature and 
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policymakers have to varying degrees treated as measures of teacher effectiveness 
(as we have already discussed in Chap. 2): namely, education, experience, content 
knowledge, time on mathematics, and instructional content. We also incorporated an 
additional teacher-level variable that was included in all iterations of TIMSS, which 
is teacher gender. 


3.2.1 Student Gender (Stmale) 


The student gender variable (Stmale) denotes the gender of each student as indicated 
by the student on the TIMSS survey. Student gender is a dichotomous variable and 
is captured by the question: “Are you a girl or boy?” Higher values indicate a male 
student. Student gender is included for both grades four and eight for each cycle of 
TIMSS included in this study (2003, 2007, 2011, and 2015). 


3.2.2 Student Language (Lang) 


The student language variable (Lang) denotes the language of each student, as 
indicated by the frequency of using the language of the test at home. Student language 
is captured by the question: “How often do you speak [language of test] at home?” 
The question has four response categories: (1) I always speak [language of test] at 
home; (2) I almost always speak [language of test] at home; (3) I sometimes speak 
[language of test] and sometimes speak another language at home; (4) I never speak 
[language of test] at home. Student language is included for both grades four and 
eight for each cycle of TIMSS included in this study (2003, 2007, 2011, and 2015). 


3.2.3 Student Estimated Number of Books in the Home 
(Books) 


The number of books in the home (Books) variable denotes the number range of 
books, not including magazines, newspapers, or school books, in each student’s home 
as estimated by the student. The variable is captured by the question: “About how 
many books are there in your home? (Do not count magazines, newspapers, or your 
school books.)” The question has five response categories: (1) None or very few (0-10 
books); (2) Enough to fill one shelf (11-25 books); (3) Enough to fill one bookcase 
(26-100 books); (4) Enough to fill two bookcases (101-200 books); (5) Enough to 


2Readers should note that, in Chap. 6, student gender, as well as several other variables, are coded 
differently in order to match the Blomeke et al. (2016) operationalization, and hence a different 
variable name is employed. Please see Chap. 6 for further details. 
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fill three or more bookcases (more than 200). As part of the TIMSS survey, each 
of the response categories included an explanatory illustration displaying how each 
category would likely look. The Books variable is included for both grades four and 
eight for each cycle of TIMSS included in this study (2003, 2007, 2011, and 2015). 


3.2.4 Teacher Experience (Exp) 


The teacher experience (Exp) variable denotes the total number of years the 
respondent reported teaching. Teacher experience is captured by the question: By 
the end of this school year, how many years will you have been teaching altogether? 
This is an open response question, and the years are reported in whole numbers. 
Teacher experience is included for both grades four and eight for each cycle of 
TIMSS included in this study (2003, 2007, 2011, and 2015). In our analyses, this 
variable is left as a simple linear variable in an effort to explore the general impact of 
teacher experience. However, as noted in Chap. 2, additional study should consider 
a non-linear relationship and/or distinguish early career teachers. 


3.2.5 Teacher Gender (Tmale) 


The teacher gender (Tmale) variable denotes the gender of each teacher as indicated 
by the teacher on the TIMSS survey. Teacher gender is a dichotomous variable and 
is captured by the question: “Are you female or male?” Teacher gender is included 
for both grades four and eight for each cycle of TIMSS included in this study (2003, 
2007, 2011, and 2015). Higher values indicate male teachers. The TIMSS survey 
coded “2” as male and “1” and female. For the purposes of this work, the variable 
has been recoded as “1” for male teachers and “0” for female teachers. 


3.2.6 Teacher Feelings of Preparedness (Prepared) 


Teacher feelings of preparedness (Prepared) is an index of teacher reported 
self-efficacy to teach mathematics. TIMSS includes a series of questions asking 
teachers the degree to which they feel prepared to teach various mathematics topics. 
These topics vary across the various cycles of TIMSS, following the structure of 
topics in that year’s mathematics framework for that grade level. For each topic, 
teachers are asked “How well prepared do you feel you are to teach the following 
mathematics topics?” and can respond on a four-point scale: not applicable (1), 
very well prepared (2), somewhat prepared (3), and not well prepared (4). For the 
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purposes of this study, we recoded these responses so that 4 = very well prepared, 
3 = somewhat prepared, 2 = not very well prepared, and | = not applicable. We 
then averaged the responses across mathematics topics. 


3.2.7. Teacher Preparation to Teach Mathematics (Mathprep) 


The Mathprep variable denotes mathematics teacher preparation by indicating 
teachers who majored in a combination of education and mathematics during their 
post-secondary studies. Mathprep is included for both grades four and eight for 
each cycle of TIMSS for which component variables of the index were available 
(2003, 2007, 2011, and 2015). Mathprep was included in the 2011 and 2015 TIMSS 
cycles, but had to be derived from the 2003 and 2007 data. The TIMSS 2015 user 
guide for the international database (Foy 2017; Supplement 3, p. 11 for grade four 
and p. 32 for grade eight) provides detailed information about how to create the 
variable Mathprep (called ATDMO5 at grade four and BTDMO5 at grade eight 
in Foy 2017). Following the coding scheme in the user guide (Foy 2017), the 
Mathprep variables for the 2003 and 2007 cycles were created similarly using “if 
then” statements to combine two variables for each grade level. For grade four, 
the first variable denoted the respondent’s post-secondary major or main area of 
study, categorized as: education = primary/elementary, education = secondary, 
mathematics, science, language of test, and other. The second variable denoted 
whether the respondent specialized in mathematics. For grade eight, the first variable 
denoted whether the respondent majored in mathematics during their post-secondary 
education. The second variable denoted whether the respondent majored in education 
with a mathematics specialty during their post-secondary education. The “if then” 
statements helped indicate 2003 and 2007 TIMSS respondents who majored in 
a combination of education and mathematics during their post-secondary studies. 
There are nuanced differences between the creation of Mathprep for grade four and 
grade eight because of differences in credentials required to teach mathematics in 
primary and secondary education. Grade four teachers often have a post-secondary 
major of primary/elementary education, whereas grade eight teachers often have a 
post-secondary major in their discipline of interest (such as mathematics). 

The new variable resulted in five categories for grade four: (1) Respondent 
did not have formal education beyond upper-secondary; (2) Respondent 
majored in something other than primary education and/or mathematics; 
(3) Respondent majored in mathematics but did not major in primary 
education; (4) Respondent majored in primary education but did not 
major or specialize in mathematics; and (5) Respondent majored in 
primary education and majored or specialized in mathematics. Similarly, 
Mathprep resulted in five categories for grade eight: (1) Respondent did 
not have formal education beyond upper-secondary; (2) Respondent majored 
in something other than mathematics and/or mathematics education; (3) Respondent 
majored in mathematics education but did not major in mathematics; (4) Respondent 
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majored in mathematics but did not major in mathematics education; and (5) 
Respondent majored in mathematics and mathematics education. 

After creating the Mathprep variable for 2003 and 2007, the 2011 and 
2015 Mathprep variable was reverse coded to match the numeric schema for 
Mathprep in 2003 and 2007. This also allowed for those who majored in primary 
education/mathematics education and also majored in mathematics to be associated 
with the highest value (5). 


3.2.8 Time Spent on Teaching Mathematics (Mathtime) 


The Mathtime variable denotes the amount of time in minutes the respondent reported 
teaching mathematics to their given class. Mathtime is included for grades four and 
eight for all years in which the variable was available (2003, 2007, 2011, and 2015). 
Mathtime was included in TIMSS for years 2003, 2007, and 2015, but in 2011, for 
both grades four and eight, the Mathtime variable was captured by two variables: 
one reporting hours (ATBMO1A) and the other reporting minutes (ATBMO1B). To 
have consistency across years, we converted the “hours” variable into minutes by 
multiplying it by sixty. Then, we added the now-converted “hours” variable to the 
“minutes” variable, resulting in one measure of time spent on teaching mathematics 
as measured by minutes. This conversion was necessary to have a consistent measure 
of time spent on teaching mathematics across grade levels and years. 


3.2.9 Teacher Curricular Alignment (Alignment) 


This variable is an index measuring the degree to which teachers are instructing 
students in the topics that are intended to be taught at that grade level according 
to national curricular expectations. The variable was therefore constructed by 
combining data from two surveys: the TIMSS teacher background survey and the 
related national background survey. TIMSS teacher respondents were presented with 
a list of mathematics topics related to the topics contained in the TIMSS framework 
for that cycle, and asked to mark whether the topic was: (1) mostly taught before 
this year, (2) mostly taught this year, or (3) not yet taught or just introduced. These 
responses were then recoded so that “mostly taught” is coded as “1” and the other 
two responses as zeros. Next, national and/or provincial administrative leaders in 
education systems participating in the TIMSS were presented with an identical list 
of topics and asked to respond at what grade level that topic should be taught by 
teachers. These responses were recoded as “1” for “O”, with “1” indicating that the 
topic should be taught at the relevant grade level (grades four and eight). The entire 
matrix of topics in that grade level was then compared with teacher responses. Cells 
were coded as “1” if a given teacher followed national curricular expectations, in 
other words, if they did teach a topic that should be taught, or did not teach a topic 
that shouldn’t be taught. By contrast, a failure to match with national curricular 
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expectations about a particular topic earned a coding of “0.” The percentage of 
alignment was then calculated, with | indicating perfect alignment and 0 indicating 
perfect non-alignment. 


3.3 Methods 


As mentioned previously, the chapters that follow use a variety of statistical methods, 
reflecting the study’s purposefully multi-model approach. The analytic strategies are 
discussed briefly here, and are addressed in more detail in the appropriate chapters. 
Unless mentioned specifically, all analyses employ the procedure for generating 
jackknifed standard errors and (when considering student outcomes) the calculation 
and combination of all plausible values, as outlined in the various TIMSS technical 
reports (see https://timssandpirls.bc.edu/isc/publications.html) and the IEA Database 
Analyzer (see https://www.iea.nl/data). 

Chapter 4 presents international averages of country-level means (including 
participating sub-units) for the predictors used throughout the remaining chapters, 
including teacher experience, teacher preparedness, teacher education (preparation 
to teach mathematics), teacher time spent on mathematics, and teacher curricular 
alignment. Variable means are calculated for each educational system participating 
in each cycle of TIMSS, and confidence intervals are used to identify statistically 
significant differences for the same country in different iterations of TIMSS. 

Chapter 5 presents a number of statistical methods that we used to examine the 
relationship between teacher characteristics and behaviors and student outcomes. We 
used a multi-model approach to test the degree to which relationships remained robust 
across countries, within countries, and across different periods, and we assessed 
the stability of estimates using different statistical techniques. As a first step, we 
compared the results for (1) single-level linear regressions (ignoring classroom-level 
clusters) of individual student outcomes as predicted by teacher variables; 
(2) two-level linear regression models that cluster students within classrooms 
using SAS PROC MIXED maximum likelihood statistical software (Singer 1998); 
and (3) a single-level linear regression of classroom means of student variables 
with teacher variables. The comparison of these analyses enables the frequency 
of statistically significant associations between purported measures of teacher 
effectiveness and student achievement in mathematics to be determined. Additional 
analyses in Chap. 5 include an examination of the stability of multilevel model 
regression coefficients within each country across the different cycles of TIMSS, and 
an exploratory fixed effects regression analysis of changes in country-level means. 

Whereas the models in Chap. 5 treat each of the teacher-level variables as 
independent variables, Chap. 6 introduces a model that uses teacher behaviors (time 
on mathematics and alignment with national curriculum standards) as mediating 
variables for instructional quality. We used a structural equation modeling (SEM) 
approach to analyzing the results for each country ina single cycle of TIMSS (namely 
2011), permitting the inclusion of additional variables like teacher professional 
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development to estimate a latent construct of “instructional quality.” A multilevel 
model clustering students within classrooms using jackknifed standard errors and five 
plausible values was applied to each educational system. Comparison of the results 
in Chaps. 5 and 6 demonstrates that a wide range of statistical techniques can be 
used to assess whether there are temporally and cross-nationally robust associations 
between measures of teacher effectiveness and student achievement in mathematics. 

Chapter 7 departs from the focus on mean student outcomes to consider the 
distributional effects of teacher effectiveness. First within-country equity was 
measured by standard deviation of pooled country-level student achievement. 
Country-level fixed effect analysis was used to assess the relationship between teacher 
effectiveness measures and student variation (measured by standard deviations). 
Second, the relationship between within-classroom variation in student outcomes 
and teacher quality was analyzed using averaged classroom-level single-level linear 
regressions. Finally, differences in teacher effectiveness between higher (top quartile) 
and low (bottom quartile) SES classrooms, as measured by using the average number 
of books in the home as a proxy for SES, were examined using Welch’s t-tests (which 
are not sensitive to sample size). This last analysis used an alternative to the jackknifed 
standard error approach (designed for the entire sample of classrooms) because it 
examined a sub-sample that is vulnerable to Type I error. 


3.4 Conclusions 


In the research presented in this volume, we use a number of teacher- and student-level 
variables and empirical approaches to examine the relationship between measures of 
teacher effectiveness and student outcomes. It should be emphasized that issues with 
cross-national comparability and the lack of truly representative student and teacher 
samples imposes some limits on these variables and statistical methods. International 
large-sample student-level longitudinal data, such as TIMSS, lacks some of the 
teacher measures now commonly used in the US-based studies, making comparisons 
with existing research difficult. A number of potentially important control variables 
(not to mention a means of distinguishing school effects from teacher effects) are 
unavailable in TIMSS. As a consequence, it is very difficult to construct more robust 
causal estimates of teacher effects in multiple countries across time. The aim of 
this study was more modest: to use multiple statistical models to explore whether 
there is a consistent pattern between measures of teacher quality (both characteristics 
and behavior) and student outcomes across time and space, and assess whether the 
ambiguous findings of US-based research were replicated on an international scale. 
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Chapter 4 ®) 
Measuring Teacher Effectiveness Across creek 
Time: What Does TIMSS Reveal About 
Education System Level Trends? 


Abstract To understand whether there are identifiable trends in teacher quality and 
instructional metrics over time, it is important to first recognize that measures of 
teacher effectiveness may vary by education system. Descriptive results for the 
teacher quality measures related to mathematics drawn from twenty years of the 
IEA’s Trends in International Mathematics and Science Study (TIMSS) data provide 
a general picture of the variation in such measures; additional consideration is given 
to student-level variables. Combining information from multiple cycles of TIMSS, 
with a specific emphasis on teacher variables, greater focus is given to the post-2003 
TIMSS data, largely because these later cycles contain additional teacher-related 
measures (such as preparedness to teach mathematics topics). Changes in education 
system means may be due to changes in underlying teacher characteristics. The data 
suggests that, among the pool of countries that participated in all the TIMSS between 
2003 and 2015, there have been increases in the international averages for the number 
of years of experience that teachers possess, their level of educational qualification to 
teach mathematics, and classroom time spent on mathematics instruction. However, 
instructional alignment with national standards has remained surprisingly modest (at 
around 55-60%) and the international average for instructional alignment declined 
for grade eight over twenty years of TIMSS. The results suggest that, in many 
countries, teachers (especially grade four teachers) retain a substantial degree of 
discretion in what to teach. Variability in teacher behaviors related to opportunity 
to learn suggests it is highly advisable to consider this factor in studies of teacher 
effectiveness. 


Keywords Opportunity to learn - Teacher education - Teacher experience - 
Teacher quality - Trends in International Mathematics and Science Study (TIMSS) 


4.1 Empirical Approach 


In this chapter we present descriptive results for educational systems (or “countries”’) 
participating all waves of the TIMSS between 1995 and 2015, with additional 
attention paid to those countries that were part of the TIMSS in every cycle since 
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2003. The focus of this chapter is on student mathematics achievement and its 
relation to factors associated with teacher quality, which we assess using both 
teacher characteristics (experience, self-efficacy, formal preparation, gender) and 
teacher behaviors (time spent on teaching mathematics, content coverage); student 
characteristics (gender, socioeconomic status, language spoken in the home) are also 
considered. Our aim was to lay the foundations for the analyses in the later chapters, 
as well as identify general patterns in teacher quality across TIMSS countries. 

Education system (henceforth referred to as country) means were calculated using 
the IEA Database Analyzer (free to access at www.iea.nl/data), with standard errors 
generated using a jackknife procedure. Because the TIMSS sampling design recruits 
a sample of representative classrooms rather than a sample of representative teachers, 
mean teacher results for each country do not necessarily reflect those of all teachers 
in a given educational system. Further, teacher-level data are not straightforward 
means, with each teacher counted equally. Rather, following the TIMSS sampling 
design, teachers are weighted according to the representativeness of their classrooms, 
based on the stratified sampling frame in each participating country. 

In this descriptive analysis, we first examined all the data across cycles! and 
then by education system. In general, between country differences were observed 
exhaustively across all variables. Within country variation over time was explored 
to identify patterns and anomalies. Only countries with three or more years of 
data were included when examining trends within countries over time. To identify 
significant differences, confidence intervals were calculated for each country and year 
combination by using standard errors. In some instances, the data were examined 
across years only to determine if there were any significant changes over time 
generally. In this chapter, we report summary frequency distributions for country 
mean values for each year and grade level (complete results are provided in Appendix 
A, and should be consulted as source data in the discussion of patterns). Where we 
discuss general patterns, we first reference all participating countries in a given year, 
rather than the common set of countries that participated every year (see Table 4.1). 
Because this could introduce bias (given changes in country participation), we also 
report mean results from a more limited common pool of countries that participated 
in every cycle of TIMSS from 2003 to 2015 (see Table 4.2) For most variables, this 
comprises 18 education systems at grade four and 26 education systems at grade 
eight (see Table 4.3). 


'This descriptive analysis focuses on data from 2003, 2007, 2011, and 2015. While data from 1995 
and 1999 are available in some instances, availability is inconsistent and thus excluded here. 
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4.2 Curricular Alignment 


Table 4.3 Educational 
systems participating in all 
cycles of TIMSS, 2003-2015 
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Grade 4 Grade 8 
Australia Australia 
Ontario (Canada) Bahrain 
Quebec (Canada) Botswana 


Chinese Taipei Ontario (Canada) 

England Quebec (Canada) 

Hungary Chinese Taipei 

Tran England 

Italy Hong Kong 

Japan Hungary 

Lithuania Iran 

Morocco Israel 

Netherlands Italy 

New Zealand Japan 

Norway Jordan 

Russian Federation South Korea 

Singapore Lebanon 

Slovenia Lithuania 

United States Malaysia 
Morocco 
Norway 


Russian Federation 


Saudi Arabia 


Singapore 


Slovenia 


Sweden 


United States 


4.2 Curricular Alignment 


4.2.1 Grade Four 


The teacher curricular alignment variable (Alignment) responses indicate that 
alignment between a country’s national expectations of topic coverage and actual 
instruction ranged between 0.34 and 0.72, where 0.00 reflects no alignment and 
1.00 reflects perfect alignment between curriculum and instruction (see Appendix 
A). This wide range suggests there is variation in curriculum alignment across 
countries and years, but warrants further exploration. The mean curricular alignment 
was 0.54, suggesting that on average, teacher instruction was aligned with the 


34 4 Measuring Teacher Effectiveness Across Time: What Does TIMSS ... 


national curriculum just over 50% of the time as an average for all TIMSS countries 
participating in the cycles from 2003 to 2015. For the subset of countries that 
participated in each year, the mean remained essentially constant (0.56 in both 2003 
and 2015). 

Examining variation in curricular alignment within a country over time provides 
interesting distinctions between educational systems. In some countries, alignment 
remained relatively constant between cycles. For example, in England, curricular 
alignment was 0.58 in 2003, 0.55 in 2007, 0.57 in 2011, and 0.55 in 2015. The 
overlapping confidence intervals suggest these differences were not significantly 
different, providing evidence that curriculum alignment in England has remained 
steady since 2003. In other countries, alignment has changed across cycles. For 
example, curricular alignment in the United States was 0.68 in 2007, 0.56 in 
2011, and 0.62 in 2015 (United States data were not available for this variable 
prior to 2007).* In this case, confidence intervals do not overlap for any cycle, 
indicating statistically significant differences in curricular alignment within the 
United States over the various cycles of TIMSS. Differences in curricular alignment 
within a country over time may be attributable to policy change, but further research 
would be needed to verify such strong conclusions. In the United States example, 
most states adopted the Common Core State Standards in the 2010-2011 school 
year (see http://www.corestandards.org/about-the-standards/development-process/), 
which may have contributed to the recorded decline in curricular alignment in 2011 
(as states and teachers adjusted to the new standards). 


4.2.2 Grade Eight 


In grade eight, there was a larger spread in the values associated with teacher 
curricular alignment than at grade four, which ranged from 0.25 to 0.88 (see Appendix 
A). Mean curricular alignment was slightly higher than that at grade four (0.59), 
meaning that, on average, instruction was aligned with the given national curriculum 
59% of the time. For the countries that participated in every cycle of TIMSS from 
2003 to 2015, the average curricular alignment declined from 0.66 to 0.58. 
Examining within country variation over time indicated that although some 
countries had constant curricular alignment over the years (i.e., Georgia), others 
showed gradual improvements in curricular alignment. For example, Australia’s 
curricular alignment has steadily improved since 2003, with curricular alignment 
measuring 0.54 in 2003, 0.55 in 2007, 0.57 in 2011, and 0.59 in 2015; note that 
the difference between their curricular alignment in 2003 and 2011 was significant. 
More often than not, curricular alignment demonstrates a more random pattern of 


? Although some countries, like the United States, do not have a national curriculum, in such 
cases national-level educational administrators (like the United States Department of Education) 
responded to the TIMSS national curriculum survey, and provided estimates of the national 
expectations for topic coverage at each grade level. 
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variation across cycles, alternating between increases and decreases. For example, 
in the United States, curricular alignment decreased in 2011 as it did at grade four; 
again this dip may be attributed to short-term policy changes, and the transition to 
the new Common Core State Standards. 

Japan showed the highest average degree of curricular alignment over time; 
alignment remained consistently above 80% in the 2003, 2007, 2011, and 2015 
cycles of TIMSS. 


4.3 Teacher Preparation to Teach Mathematics 


4.3.1 Grade Four 


The teacher preparation to teach mathematics (Mathprep) variable for grade four 
ranged from 1.00 to 4.93 (see Appendix A). This variable was measured by a 
five-point scale, where 1.00 corresponds with having no formal preparation to 
teach mathematics and 5.00 corresponds with having specialized preparation in both 
primary education and mathematics (namely that a given primary teacher would have 
been trained for content knowledge (CK), pedagogical content knowledge (PCK), 
and general pedagogical knowledge appropriate for grade four students). A mean 
value of 3.67 indicates that, on average, across all cycles and countries, teachers 
majored in primary education and/or mathematics but did not always have both 
qualifications. For the pool of 18 educational systems that participated in all four 
cycles of TIMSS from 2003 to 2015, the mean was 3.59 across years, rising steadily 
from 3.41 to 3.74. 

In general, values for teacher preparation to teach mathematics at grade four 
remained relatively consistent within countries throughout the testing years, with 
only slight variations within some countries. For example, Quebec’s values for this 
variable were 3.86, 3.89, 3.95, and 3.95 for the years 2003, 2007, 2011, and 2015, 
respectively. While these values slightly increased over the years, the differences were 
not significant, which suggests that there was little change in teacher preparation to 
teach mathematics at grade four in Quebec. Conversely, as an example an education 
system that showed significant changes in teacher preparation to teach mathematics 
at grade four, in Singapore the value went from 3.55 in 2003 to 4.18 in 2015. 
This significant difference indicates that teachers in Singapore have become better 
prepared to teach mathematics in grade four. Significant differences in teacher 
preparation to teach mathematics within countries, as in Singapore, may reflect policy 
changes that impacted teacher preparation, meriting further exploration. 

Notably, Italy had consistently low values for teacher preparation to teach 
mathematics in grade four, of <1.50 for all four cycles of TIMSS. At the opposite 
end of the spectrum, in the Netherlands, teacher preparation to teach mathematics 
values were >4.5 for three out of the four TIMSS cycles considered. 
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4.3.2 Grade Eight 


The findings for teacher preparation to teach mathematics in grade eight were very 
similar to those for grade four, ranging between 1.00 and 4.89 with a mean of 
3.69, which suggests that, on average, grade eight teachers majored in mathematics 
education and/or mathematics, but did not always have both qualifications (see 
Appendix A). Again, this TIMSS scale presumes that more exhaustive formal 
preparation in mathematics content and pedagogy indicates better preparation to 
teach mathematics. The mean for commonly participating countries (Table 4.3) was 
3.60, increasing from 3.52 in 2003 to 3.79 in 2015. 

Throughout the testing years, there was more within country variation in 
teacher preparation to teach mathematics at grade eight than at grade four. Teacher 
preparation to teach mathematics in grade eight significantly increased between 2003 
and 2015 in some countries (such as Quebec, Canada), and significantly decreased 
in others (for example, Saudi Arabia), or showed no consistent pattern (for example, 
England). The greater temporal within-country variation in teacher preparation to 
teach mathematics in grade eight points to possible between grade differences in 
teacher preparation requirements. For example, perhaps the requirements around 
teacher preparation to teach mathematics in grade four are more clearly defined than 
they are for grade eight. 

At grade eight, Morocco consistently had low values associated with teacher 
preparation to teach mathematics (three reporting years with values <2.00), while 
conversely, Romania had three reporting years with values >4.00. 

Interestingly, at both grades four and eight, the mean values associated with 
teacher preparation to teach mathematics increased over time. At grade four, the 
mean value for teacher preparation to teach mathematics was 3.53 in 2003 and 3.85 
in 2015. Again, at grade eight, the mean value was 3.69 in 2003 and 3.88 in 2015. 
Similar results were found when trends were restricted to only those countries that 
participated in all four cycles of TIMSS between 2003 and 2015 (an increase of >0.3 
for both grades). These increases may point to overall improved teacher preparation to 
teach mathematics at grades four and eight across countries. One possible explanation 
for this improvement is that teacher preparation became more of a policy priority 
over time, resulting in more stringent teacher preparation requirements. 


4.4 Teacher Time on Mathematics 


4.4.1 Grade Four 


The large range (from 65.90 to 479.24 min) of teacher reported time spent on teaching 
mathematics as measured in minutes at grade four (Mathtime) suggests significant 
variation within counties over time (see Appendix A). For many countries there were 
differences in time spent on teaching mathematics across the different test cycles. 
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For example, Australia’s average teacher time on mathematics started at 260 min in 
2003, remained steady in 2007 at 266 min, increased dramatically to 346 min in 2011, 
and finally decreased to 306 min in 2015. The changes in 2011 and 2015 were both 
significant, and may reflect the introduction of a national curriculum between 2008 
and 2012. Similarly, time on mathematics in the United States started at 245 min 
in 2003, and subsequently increased to 289 min in 2007, 343 min in 2011, and 
359 min in 2015. These increases were all statistically significant, with the largest 
jump occurring between 2007 and 2011. As mentioned earlier, this increase in the 
time teachers spent on teaching mathematics in the United States may be attributed to 
implementation of the Common Core State Standards. There was a general increase in 
time spent on mathematics in fourth grade among the pool of commonly participating 
countries (Table 4.3), rising from 242 min in 2003 to 264 min in 2015 (although this 
was a decrease from a high of 277 min in 2011). 

The wide within-country variation across years may suggest that time spent 
teaching mathematics in grade four is not often standardized. However, there were 
a handful of instances where teacher time on mathematics was consistent within 
countries over years, such as in Singapore where time spent on mathematics varied 
only between 325 and 329 min over five testing periods. 

Countries consistently at the lower end of the grade four time on mathematics 
spectrum included Chinese Taipei, Norway, and Sweden; countries consistently at 
the higher end of the grade four time on mathematics spectrum included Portugal, 
Canada (Quebec), Italy, and the United States Taken together, these findings suggest 
wide variations in time on mathematics in grade four. 


4.4.2 Grade Eight 


While the range (80.60-350.35 min) of teacher reported time spent on teaching 
mathematics as measured in minutes in grade eight was not quite as large as grade 
four, it was still large enough to warrant further exploration (see Appendix A). There 
was a weaker trend among commonly participating TIMSS countries (Table 4.3) 
between 2003 and 2015, with the average time spent on teaching mathematics 
increasing from 212 to 220 min, although, as with grade four, the largest averages 
were found in 2011. 

Examining within country variation provides a clearer story about how much 
time teachers spend teaching mathematics in grade eight. There was still variation 
in teacher time on mathematics in grade eight, but the variation was not as large as 
observed for grade four. Recall that Australia’s average time on mathematics in grade 
four ranged from 260 to 346 min over four test cycles. For grade eight, Australia’s time 
spent on teaching mathematics varied between 208 and 220 min over those same four 
test cycles. Similarly, in the United States, while there were still significant increases 
in grade eight time spent on mathematics over the test cycles, the increases were not 
as large as they were in grade four. For example, over four test cycles, the grade four 
time on mathematics in the United States varied from 245 to 359 min, a sweep of 
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over 100 min; meanwhile, the grade eight time on mathematics in the United States 
over the same four test cycles ranged only between 226 and 265 min. These differing 
degrees of variation within countries for time on mathematics at grades four and 
eight may suggest there are widespread grade-level differences in how much time 
teachers spend on teaching mathematics. 

Countries consistently at the lower end of the grade eight time on mathematics 
spectrum included Cyprus, the Netherlands, Japan, and Sweden; countries 
consistently at the higher end of the grade eight time on mathematics spectrum 
included Lebanon, Canada (Ontario), Chile, and the United States. 


4.5 Teacher Preparedness 


4.5.1 Grade Four 


Teacher feelings of preparedness to teach mathematics (Prepared) were measured 
on a four-point scale, where higher values indicated teachers felt better prepared 
to teach mathematics. At grade four, teacher feelings of preparedness ranged from 
2.07 to 3.88, with a mean value of 3.24 (see Appendix A). For those countries that 
participated in each cycle of TIMSS between 2003 and 2015 (Table 4.3), the mean was 
3.16 (rising sharply between 2003 and 2007, but remaining stable afterwards). This 
mean value suggests that generally teachers feel well prepared to teach mathematics. 

Some recognizable patterns emerge when examining grade four teacher feelings 
of preparedness within countries over the different testing years. In many countries, 
the values associated with grade four teacher feelings of preparedness significantly 
increased between the 2003 and 2007 test cycles, and then remained steady for the 
remaining test cycles. For example, the grade four teacher feelings of preparedness 
in Norway were 2.61 in 2003, 3.69 in 2007, 3.51 in 2011, and 3.78 in 2015. Similar 
increases between 2003 and 2007 were seen in several other countries, such as 
Australia, Belgium (Flemish), Canada (Ontario), Canada (Quebec), Hungary, Italy, 
Morocco, and the United States. This indicates some change around teacher feelings 
of preparedness occurred between 2003 and 2007. One possible explanation is that 
TIMSS does not ask about the same mathematics topics in every cycle; there are 
cycle-to-cycle alterations to the TIMSS framework that mean that the same topics 
are not asked every time. It is thus possible that differences in mean teacher responses 
could be partly attributable to the survey instrument rather than the underlying 
construct. 


4.5.2 Grade Eight 


For grade eight, there was a wider range for teacher feelings of preparedness than seen 
at grade four, with values from 1.06 to 3.91 and an overall mean value of 2.91 (see 
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Appendix A). The average of 2.91 may suggest that generally grade eight teachers 
felt adequately prepared to teach mathematics, but their feelings of preparedness are 
not as strong as those of the grade four teachers. However, when the pool of education 
systems was restricted to only those that participated in all cycles of TIMSS between 
2003 and 2015, the international mean was 3.17, virtually indistinguishable from 
that reported for grade four teachers. 

As with grade four, the values associated with grade eight teacher feelings of 
preparedness significantly increased between the 2003 and 2007 test cycles, and then 
remained steady for the remaining test cycles. For example, the grade eight teacher 
feelings of preparedness in the United States were 2.84 in 2003, 3.87 in 2007, 3.66 
in 2011, and 3.48 in 2015. Many other countries displayed this pattern when looking 
at grade eight teacher feelings of preparedness and, again, this pattern may indicate a 
systemic change in how TIMSS measured this variable. By contrast, at grade eight, 
Denmark reported high values (>3.66) for teacher feelings of preparedness for three 
consecutive test cycles. 

At both grades four and eight, the mean values associated with teacher feelings 
of preparedness displayed large increases between 2003 and 2007, and levelled off 
in subsequent cycles of TIMSS. For example, at grade four the overall mean value 
for teacher feelings of preparedness was 2.52 in 2003 and 3.38 in 2007. Similarly, 
at grade eight the overall mean value was 2.58 in 2003 and 3.58 in 2007. It seems 
unlikely that teacher feelings of preparedness would change so considerably between 
two consecutive test cycles, suggesting that these increases may also reflect changes 
in the metric of teacher feelings of preparedness (due to alterations in the TIMSS 
framework); this would affect all grade levels and education systems. 


4.6 Teacher Experience 


4.6.1 Grade Four 


The teacher experience variable captures the total number of years the teacher 
has been teaching (Exp). At grade four, teacher experience ranged from 7.63 to 
27.64 years, with a mean reported value of 16.29 years (see Appendix A). The 
comparable mean for the restricted sample of eighteen countries was 16.79 years, 
rising from 15.92 years in 2003 to 17.65 years in 2015, suggesting an increase in 
teacher experience over time. 

When looking at teacher experience within countries over years, there were some 
countries where the amount of reported teacher experience was consistent and some 
countries that showed significant variation in reported teacher experience. Teacher 
experience in the United States was fairly consistent, hovering between a mean of 13 
and 14 years of experience across the test cycles from 2003 to 2015. However, the 
2015 mean value of 13.13 years was significantly lower than the other values. This 
slight decrease in mean years of teacher experience may coincide with the economic 
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crisis the United States experienced in the late 2000s. During this time, many older 
teachers opted for retirement options, and this would have systematically driven 
down mean teacher experience as measured in years. Conversely, mean grade four 
teacher experience appeared quite variable in Ontario, Canada. While the values for 
Ontario were similar in 2003 and 2007 (13.11 and 13.15 years, respectively), mean 
teacher experience dropped to 11.52 years in 2011 and then substantially increased to 
14.99 years in 2015. Further research is required to better understand such significant 
variations. 

The education systems that consistently reported the lowest number of years 
of teacher experience at grade four included Kuwait and Singapore; both reported 
multiple values <10 years. Those that consistently reported the highest number 
of years of teacher experience at grade four included Lithuania and Georgia; both 
reported multiple values >20 years. 


4.6.2 Grade Eight 


At grade eight, teacher experience ranged from 5.00 to 26.72 years, with a mean 
of 15.77 years (see Appendix A). In other words, grade eight teachers who teach 
mathematics have 15.77 years of teaching experience on average across test cycles, 
as measured by TIMSS. The comparable mean for the restricted sample of eighteen 
countries was 15.21 years. The number of years of teacher experience reported by 
grade eight teachers started at a lower level and ended at a lower level than grade four, 
suggesting there were between-grade differences in the level of teacher experience. 
The trends suggested modest change in net experience, increasing from 15.01 years 
in 2003 to 15.78 years in 2015. 

The within-country analysis of grade eight teacher experience also revealed 
similar findings to grade four; some countries displayed consistency in reported 
teacher experience at grade eight and some countries displayed variation in reported 
teacher experience at grade eight. For example, teacher experience in Australia, Italy, 
and the United States was relatively consistent across cycles at grade eight, while 
there was more variation in reported teacher experience at grade eight in Chile, Egypt, 
and Chinese Taipei. 

The education systems that consistently reported the lowest number of years 
of teacher experience at grade eight included Ghana and Botswana; both reported 
multiple values <nine years. Those that consistently reported the highest number 
of years of teacher experience at grade eight included Romania and the Russian 
Federation; both reported multiple values >20 years. 
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4.7 Teacher Gender 


4.7.1 Grade Four 


Teacher gender is a dichotomous variable; a response of 0.00 denotes the teacher 
is female and a response of 1.00 denotes the teacher is male (Tmale). For grade 
four, the teacher gender variable ranged between 0.00 and 0.80, with a mean of 0.21 
(see Appendix A). This mean value is quite informative because an average close to 
0.00 indicates teachers tend to be female, a mean of 0.50 indicates male and female 
teachers are equally represented, and a mean close to 1.00 indicates teachers tend 
to be male. The grade four mean of 0.21 (0.20 for the restricted sample), which 
holds over time, indicates that TIMSS respondents were more likely to be female. 
In general, teaching tends to be a female-dominated profession in many countries 
(especially at earlier grades), so these results are not surprising. 

Between-country findings yielded interesting results related to grade four teacher 
gender. Most countries’ means for this variable were closer to 0.00 across test cycles, 
indicating that teachers in each of the countries tended to be female. However, a few 
countries had results that suggested teacher representation was more gender balanced. 
Denmark, for example, reported teacher gender values of 0.51 in 2007, 0.42 in 2011, 
and 0.47 in 2015. In Denmark there are almost equal numbers of male and female 
teachers, which may reflect Denmark’s reputation for greater gender equity. In a 
few countries, the results indicated that teachers were more often male. In Yemen, 
for example, teacher gender values were 0.74 in 2003, 0.74 in 2007, and 0.78 in 
2011; one explanation for this might be that the culture in Yemen supports a more 
male-dominated workforce, resulting in fewer female employees in general. 

In general, the within-country analysis uncovered little variation in grade four 
teacher gender across years. However, a few countries did see decreases in their 
teacher gender variable over time. For example, Morocco’s values for grade four 
teacher gender were 0.64 in 2003, 0.50 in 2007, 0.50 in 2011, and 0.37 in 2015. 
The substantial change from 2003 to 2015 implies teachers were more likely to be 
males in 2003, but, by 2015, teachers were more likely to be female. One possible 
explanation for this shift is that, over time, it became more socially acceptable for 
females to hold teaching positions. 


4.7.2 Grade Eight 


The descriptive statistics for teacher gender in grade eight differ from those for grade 
four. At grade eight, teacher gender varied between 0.00 and 1.00, with a mean of 
0.42 for both the overall sample and the more restricted sample of eighteen countries 
(see Appendix A). This mean holds over time and implies that teachers were more 
often female, and also that for grade eight the gender distribution was more equal 
between males and females than at grade four. This also aligns with our general 
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conception that teachers in elementary grades are more likely to be female, whereas 
teachers in upper elementary and high school are more evenly distributed between 
males and females. 

While many countries, such as the United States, had consistent values for grade 
eight teacher gender over time, others, like Japan, displayed a level of variation. 
Generally, there appeared to be more within-country variation for grade eight teacher 
gender than there was at grade four. 

For all four cycles of TIMSS that we investigated, Ghana, Morocco, and Japan 
consistently reported having more male teachers than female teachers, whereas the 
Russian Federation, Latvia, Lithuania, and Georgia reported that their teachers were 
almost exclusively female teachers. 


4.8 Student Performance 


4.8.1 Grade Four 


At grade four, student mathematics performance (Performance) in TIMSS ranged 
from a point score of 223 to 618, with a mean of 491 (see Appendix A). Examining 
the international mean over time suggests slight increases in overall performance 
over time. For example, at grade four, the overall mean student performance was 490 
in 2003; by 2015, the mean had increased to 506. For our more restricted sample of 
countries, the international mean rose from 505 in 2003 to 527 in 2015. 

Cross-national comparison of grade four student performance within countries 
showed that some countries demonstrated considerably more variation than others, 
which may merit deeper investigation. For example, in Armenia, the mean grade 
four student performance was 455.92 in 2003, 499.51 in 2007, and 452.28 in 2011. 
These back and forth changes suggest a degree of instability in grade four student 
performance between test cycles in Armenia. Other countries with considerable 
variation included Qatar, Yemen, and Kuwait, to name a few. At the same time, 
Australia saw very little variation in their mean grade four student performance over 
time, with scores of 499 in 2003, 516 in 2007, 516 in 2011, and 517 in 2015. Other 
countries exhibiting similar consistency include Belgium (Flemish), New Zealand, 
and Italy. 

As has been widely noted in scholarly and popular publications, countries 
with the highest scores over time are largely located in East Asia, while countries 
with the lowest scores over time are largely located in West Asia and Africa. As 
such, geographical and cultural differences may play an important role in student 
achievement. 


4.8 Student Performance 43 


4.8.2 Grade Eight 


At grade eight, the range in student performance scores over time and across countries 
varied between 264 and 621 score points, with an international mean of 475 (see 
Appendix A). The examination of the mean over time yields similar findings to that 
for grade four, with the mean slightly increasing from 468 in 2003 to 481 in 2015, 
and from 490 in 2003 to 501 in 2015 for our more restricted sample (Table 4.3). In 
other words, overall TIMSS performance has improved over the test cycles that we 
considered in our analysis. 

In general, there was greater variation in the within-country analysis of student 
performance at grade eight than at grade four. For example, changes in Chile’s grade 
eight student performance scores were substantial, with a score of 387 in 2003 rising 
to a score of 427 in 2015. This increase aligns with the overall international increase 
in TIMSS performance over time. Meanwhile, Malaysia saw a drop in grade eight 
student performance, their mean score being 508 in 2003 and 465 in 2015. Australia 
was one of the few countries whose student performance scores were consistent over 
time at grade eight. 

As we found for grade four, countries with the highest grade eight student 
performance over time were largely located in East Asia, while countries with the 
lowest grade eight student performance over time were often located in West Asia 
and Africa. 


4.9 Books in the Home 


The number of books in the home (Books) is a control variable that serves as a proxy 
variable to indicate student socioeconomic status. In the TIMSS survey, students are 
asked to estimate the number of books in their home, with responses placed ona | to5 
scale. A larger value denotes more books in the home, which generally corresponds to 
higher socioeconomic status. Values for books in home looked similar across grades 
four and eight, ranging between 1.61 and 4.04 at grade four, and 1.84 and 4.31 at grade 
eight (Appendix A). The international mean across cycles for books in home was 2.82 
at grade four and 2.81 at grade eight. International means were quite similar for our 
more restricted sample of 18 countries, being 2.85 at both grade four and grade eight. 
Within-country variation across cycles was minimal, indicating that there was little 
change in socioeconomic conditions across the TIMSS administrations. In general, 
countries with higher values for books in home were the wealthier countries, whereas 
countries with lower values for books in home were less wealthy countries. 
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4.10 Student Language 


Student language (Lang) is a control variable that captures the alignment between 
the language the test is delivered in and how often that same language is spoken 
in the student’s home. This variable was measured on a four-point scale. A lower 
value indicates more overlap between the language of the test and language spoken 
at home, for example, a value of 1.00 denotes that the student always speaks the 
language of the test at home, whereas a value of 4.00 means that the student never 
speaks the language of the test at home. Values for student language at grade four 
ranged between 1.03 and 3.08, with a mean of 1.47 (or 1.49 in our more restricted 
sample of 18 countries; Table 4.3). The distribution of student language indicated 
that, in most instances, the language of the test was always or almost always spoken 
at home. Grade eight results were quite similar, with a mean of 1.52 (1.57 for the 
restricted sample of commonly participating countries; Table 4.3). 


4.11 Conclusions 


With respect to Research Question 1, “Are there identifiable trends in teacher 
quality and instructional metrics over time ?”, the results to this chapter indicate that 
while trends vary from country to country, a focus on a common pool of countries 
(Table 4.3) suggested substantial change in teacher quality metrics. Specifically, at 
both grades four and eight, there were broad-based increases in teacher education 
and teacher experience. There was also an increase in time spent on mathematics 
in grade four over the cycles of TIMSS that we investigated, and a smaller increase 
at grade eight. By contrast, teacher self-reported preparedness to teach mathematics 
has been largely stable since 2007. Alignment of instructional content with national 
expectations has also stayed at a relatively consistent level at grade four, but 
alignment has declined since 2003 at grade eight. 

The most striking finding of this chapter was the very modest degree of alignment 
between teacher instructional content and national expectations of content coverage 
in mathematics. Among those educational systems reporting alignment in all four 
TIMSS cycles that we investigated, the international mean was only 0.55 at grade 
four and 0.60 at grade eight. At grade four, only two educational systems, Hong 
Kong and Korea, exhibited instructional alignment >70% over the last four cycles 
of TIMSS. High alignment rates were much more common at grade eight. It was 
further quite surprising to find that the United States, which has no official national 
curriculum, scored relatively highly for alignment. Overall, the results suggest that, 
in many countries, teachers (especially grade four teachers) maintain substantial 
discretion in what to teach, a conclusion bolstered by the similarly strong variation 
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in time spent on mathematics. Compared with the much more stable teacher 
characteristics (experience, feelings of preparedness, and level of formal education) 
reported in this chapter, the reported variability in teacher behaviors related to 
opportunity to learn demonstrates the importance of incorporating these factors into 
studies of effectiveness. 
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Chapter 5 M®) 
Teacher Quality and Mean Student cro 
Outcomes: A Multi-model Approach 


Abstract IEA’s Trends in International Mathematics and Science Study (TIMSS) 
can potentially be used to address important questions about the role of teachers 
in influencing student outcomes. To establish the potential relationships between 
student achievement and different types of teacher quality and instructional metrics 
and how stable these relationships may be across time, a variety of different statistical 
approaches were implemented and compared. The results from a single-level 
(unclustered) model, a multilevel model, and a classroom-mean regression model 
were compared, as was the stability of within-country estimates across time. The 
potential utility of country-level fixed-effects analysis was also explored. The 
complex sampling design of TIMSS enables robust standard errors to be generated. 
The analyses suggested only a weak relationship between teacher quality measures 
and student outcomes across countries and grade levels, and considerable instability 
in within-country estimates across time, especially at grade eight. 


Keywords International large-scale assessment (ILSA) - Opportunity to learn - 
Statistical methodology - Teacher education - Teacher experience - Teacher 
quality - Trends in International Mathematics and Science Study (TIMSS) 


5.1 Introduction 


To date, research into relationships between teacher characteristics and student 
outcomes has relied on data from only a few countries, or been restricted to analysis 
of data collected at single point in time, or both. Such limitations have made it 
difficult to draw general conclusions about whether particular measures of teacher 
quality (such as, instructional alignment) are systematically associated with higher 
learning gains, or whether any such gains are due to other factors or the unique 
circumstances of a given educational system. In this chapter, we take advantage of 
the measures of teacher quality within the TIMSS framework, and the extensive 
international data collected by TIMSS over 20 years, to investigate this problem 
more thoroughly, using a variety of statistical methods. Our aim was twofold: first, 
to identify whether given teacher-related metrics were generally related to student 
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mean outcomes within and across countries; and second, whether these relationships 
were sensitive to the statistical method employed or a particular sample (namely, data 
collected from one cycle of TIMSS). This chapter can be considered to be an extended 
robustness check on the association between teacher quality and student mathematics 
performance, comparing particular results to different educational contexts (namely 
differing national education systems), different years, and different methodological 
choices. 

A secondary consideration was the challenge of balancing rigorous quantitative 
methods with the practical problems of dealing with large-scale datasets. 
Sophisticated multilevel models with very large sample sizes, such as would be 
necessary if incorporating all of the TIMSS countries in every cycle into one model, 
can be computationally quite demanding, even with modern computing power. More 
sophisticated methods would likely identify more precise and less biased estimates. 
This chapter presents an exploratory study, aiming to assess whether there were 
obvious patterns across time and space, preliminary to more detailed research. 

We begin by assessing how frequently the relationship between measures of 
teacher effectiveness and student outcomes were statistically significant across 
countries. As noted in Chap. 2, the existing research literature has failed to identify 
any consistent relationships between teacher characteristics (experience, education, 
teacher content knowledge) and student outcomes, but has indicated that student 
outcomes may be more strongly associated with teacher behaviors (content coverage 
and time on teaching mathematics). Thus, we aimed to determine whether this was 
a consistent finding across different educational systems, and how sensitive these 
findings were to the statistical method used. To do this, we compared the results of 
a “full” model, incorporating teacher- and student-level effects, with those from a 
model that ignored student clustering within classrooms and classroom-level means. 

Having considered the effect of the different statistical methods on our findings, 
we next investigated the stability of these associations across time. Although the 
country-level averages of teacher quality measures may vary over time, the direction 
and strength of relationship between teacher and student factors should be more 
consistent, assuming low measurement error, sufficiently sensitive and reliable 
instrumentation, and rough institutional stability in a particular educational system. 
We analyzed the stability of statistical estimates across time, first by comparing 
multilevel regression coefficients for a given country across multiple years of the 
TIMSS, and next by conducting a fixed-effect analysis using country-level means. 
Finally, we assessed the robustness of multilevel model results by comparing a 
simplified means of calculating standard errors with the more elaborate jackknifing 
procedure recommended by the various TIMSS user guides. 

All of the analyses in this chapter use a basic additive model in which student 
mean achievement in mathematics is predicted by six teacher variables (experience, 
preparedness, education, alignment, time spent on mathematics, and teacher gender) 
and three student control variables (books in the home, language spoken in the 
home, and student gender). The operationalization of these variables has already been 
discussed in Chap. 3. We applied this model to each education system participating 
in TIMSS for every cycle of participation, using standard jackknifed standard errors 
and five plausible values. Education systems that did not include one of the five main 
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teacher variables (experience, education, preparedness, alignment, and time spent on 
mathematics) were excluded from the analysis. Each education system was analyzed 
separately by year and grade level. 

The analysis presented in the main text is summary data combining the results for 
multiple educational systems. (For detailed country-level results for each statistical 
model, please consult Appendix B.) 


5.2 Consistency Across Pooled, Multilevel, 
and Classroom-Means Models 


We tested the basic linear model using three different statistical models. The first is a 
simple pooled within-country model using ordinary least squares (OLS) regression. 
The second is a multilevel model that clusters students within classrooms, with 
student variables at level one and teacher variables at level two. The third model 
aggregates student-level variables at the classroom level to create classroom means 
(in other words, classrooms rather than students are the unit of analysis). These 
classroom-mean results are analyzed using a single-level model. 

We focused on five teacher-level variables: teacher experience (Exp), teacher 
education to teach mathematics (Mathprep), time spent on teaching mathematics 
(Mathtime), instructional alignment with national standards (Alignment), and 
self-reported preparedness to teach mathematics (Prepared). Our analyses included 
participating TIMSS education systems where all these variables were available and 
excluded systems where some variables were unavailable. Consequently, we applied 
our three types of model (pooled, multilevel, and classroom means) to 307 cases at 
both grades four and eight for the 2003, 2007, 2011, and 2015 cycles of TIMSS. 
Each of the 307 cases represents an educational system in a single cycle of TIMSS. 
With three statistical models per case, this comprises 921 separate regressions, with 
1535 teacher effectiveness variables compared across models (307 x 5 = 1535) (see 
Table 5.1). 

In terms of the consistency of statistical inference across all three models, our 
results showed that, among the 1535 comparisons, 1151 (75.0%) produced similar 
estimates, either significant or non-significant, across the pooled, multilevel, and 
classroom-means models. Among those comparisons with consistent estimates of 
significance, 78 (5.1%) comparisons were significant with same direction, and 1073 
(69.9%) cases were statistically non-significant (p > 0.05). That is, although the 
estimates of regression coefficients and the standard errors were slightly different 
across the three models, two-thirds of them were substantively identical. In over 
two-thirds of comparisons, none of the models identified a statistically significant 
effect of teacher effectiveness measures on student outcomes. 

There was one exceptional case where all three model estimates were statistically 
significant, but in opposite directions. This unusual case considered alignment 
of the grade eight curriculum in Malta in the 2007 cycle of TIMSS; coefficient 
estimates for the single-level and classroom-means models were 160.3 and 103.5, 
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Table 5.1 Number of model Grade Cycle of Number of Number of 
segressions by grade-and TIMSS education model? 
cycle of TIMSS systems with | regressions 
required data 

Grade four 2003 12 36 

Grade four 2007 35 105 

Grade four 2011 48 144 

Grade four 2015 45 135 

Grade eight | 2003 31 93 

Grade eight | 2007 50 150 

Grade eight | 2011 45 135 

Grade eight | 2015 41 123 

Total 307 921 


Notes *Models may be pooled single-level models, multilevel 
models, or classroom-means models 


respectively (positive and statistically significant), however, the coefficient estimate 
for the multilevel model was —33.0 (negative and statistically significant). 

We also compared the three models in pairs to assess the statistical significance and 
directional discrepancy of the regression coefficient estimates. This approach looked 
at partial consistency among two statistical models, rather than across all three. 
Sixty-one (4.0%) comparisons were statistically significant with same direction for 
both the single-level and multilevel model, but not for classroom-means model. For 
instance, the estimates of grade four teacher experience in Iran for 2003 using the 
single-level and multilevel models were 1.2 and 1.5, respectively, but the estimate 
of the same predictor using the classroom-means model was —0.6, which was not 
statistically significant. 

In some cases, even though both the single-level and multilevel models produced 
statistically significant estimates, the directions opposed each other. For example, 
when analyzing grade eight data for Jordan in 2007, while the estimate of 
preparedness in single-level model was 32.1, it was —9.9 using the multilevel 
model. Similarly, grade eight teacher preparedness for Tunisia in 2007 was 8.7 with 
single-level model, but —5.5 with the multilevel model. 

When making comparisons between the multilevel and classroom-means models, 
there were 23 (1.5%) comparisons where the estimates of both models were 
statistically significant with same direction, where this was not true for single-level 
model. For instance, the estimate of grade four teacher experience for Chinese Taipei 
in 2003 was identical (0.3) across the three models, but this was not significant for the 
single-level model (the magnitude of the estimate was small, however). There was 
also one exceptional case where estimates of time spent on teaching mathematics 
were significant for both models but in opposite directions. For Iran in 2015 at grade 
eight, the time spent on teaching mathematics had a positive coefficient of 0.1 in the 
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multilevel model, but a coefficient of —0.1 for the classroom-means model. However, 
given the very small effect size, this inconsistency should not be overstated. 

There were more consistent results between the single- and classroom-means 
models. We found that there were 52 (3.4%) comparisons that were statistically 
significant and in the same direction, where this was not the case for multilevel 
model. Interestingly, there were no comparisons between the single-level and 
classroom-mean models that showed opposing directions. However, there were 125 
(8.1%) comparisons where the estimates were significant only for the multilevel 
model. 

Focusing on our five teacher-level variables, coefficients differed across the three 
models with regard to statistical significance for each variable. On examination of the 
307 cases (countries per year) for each teacher effectiveness variable, we were able to 
classify the results into eight categories: (1) significant for only the multilevel model, 
(2) non-significant for only the multilevel model, (3) significant for both 
the single-level and multilevel models, (4) significant for both the multilevel 
and classroom-means models, (5) non-significant for both multilevel and 
classroom-means models, (6) non-significant for both single-level and multilevel 
models, (7) significant for all three models, and (8) non-significant for all three 
models (Fig. 5.1). 

Our analysis strongly suggests that, overall, measures of teacher effectiveness have 
a weak and inconsistent association with student outcomes. The vast majority of the 
time (66-75% of the time), the three statistical models were in agreement that there 
was no Statistically significant relationship with mean mathematics performance for 
any measure of teacher effectiveness. 
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Fig. 5.1 The number of significant (SIG) and non-significant (INSIG) associations between teacher 
variables and student outcomes by educational system using single-level (SLM), multilevel (MLM), 
and classroom-means (CLM) models, based on data from TIMSS 2003-2015 
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When we considered specific teacher-level variables, for teacher experience there 
were 25 of the 307 cases (8.1%) where the estimates were statistically significant 
for only the multilevel model. In contrast, five (1.6%) cases of the estimates were 
statistically non-significant for only the multilevel model. Regarding significance in 
pairs, there were 19 (6.2%) cases where the estimates were statistically significant for 
both the single-level and multilevel models. We also found out that seven (2.3%) cases 
were Statistically significant for only the multilevel and classroom-means models. 
Conversely, in terms of non-significance in pairs, in 13 (4.2%) cases the estimates 
were non-significant for both the multilevel and classroom-means models, and in six 
(2.0%) cases the estimates were non-significant for both single-level and multilevel 
models. Lastly, with respect to consistency across three models, in 19 (6.2%) cases 
the estimates were statistically significant across all three models. In the remaining 
213 (69.4%) cases the estimates were non-significant for all three models. 

For teacher education to teach mathematics (Mathprep), the estimates were 
statistically significant for only the multilevel model in 22 (7.2%) of the 307 
cases. In contrast, there were 13 (4.2%) cases where the estimates were statistically 
non-significant for only the multilevel model. Regarding significance in pairs, the 
estimates were statistically significant for only the single-level and multilevel models 
in 13 (4.2%) cases. We also found that only three (1.0%) cases were statistically 
significant for only the multilevel and classroom-means models. Conversely, in terms 
of non-significance in pairs, in 11 (3.6%) cases the estimates were non-significant for 
both the multilevel and classroom-means models, and there were 10 (3.3%) cases 
where the estimates were non-significant for both the single-level and multilevel 
models. Lastly, in six (2.0%) cases the estimates were statistically significant across 
all three models, while in the remaining 229 (74.6%) cases the estimates were 
non-significant for all three models. 

For time spent on teaching mathematics (Mathtime), the estimates were 
statistically significant for only the multilevel model in 26 (8.5%) of the 307 cases. 
In contrast, the estimates were statistically non-significant for only the multilevel 
model in eight (2.6%) cases. Regarding significance in pairs, there were seven (2.3%) 
cases where the estimates were statistically significant for both the single-level and 
multilevel models. We also found that only six (2.0%) cases that were statistically 
significant for both the multilevel and classroom-means models. Conversely, in 
terms of non-significance in pairs, 17 (5.5%) estimates were non-significant for 
both the multilevel and classroom-means models, and 10 (3.3%) estimates were 
non-significant for both the single-level and multilevel models. Lastly, considering 
consistency across three models, 28 (9.1%) estimates were statistically significant 
across all three models and the remaining 205 (66.8%) estimates were non-significant 
for all three models. 

For teacher alignment with national standards, estimates were statistically 
significant for only the multilevel model in 20 (6.5%) of the 307 cases. In contrast, 
the estimates were statistically non-significant for only the multilevel model in eight 
(2.6%) cases. Regarding significance in pairs, there were 13 (4.2%) cases where 
the estimates were statistically significant for both the single-level and multilevel 
models. We also found that only five (1.6%) cases were statistically significant 
for both the multilevel and classroom-means models. In terms of non-significance 
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in pairs, eleven (3.6%) estimates were non-significant for both the multilevel and 
classroom level model, and in eight (2.6%) cases the estimates were non-significant 
for both the single-level and multilevel models. Lastly, 20 (6.5%) of the estimates 
were Statistically significant across all three models, although, among these 20 cases, 
as we mentioned previously, for the grade eight curriculum in Malta in the 2007 
cycle of TIMSS, the estimates for the single-level and classroom-means models were 
positive while the estimate for the multilevel model was negative. The remaining 222 
(72.3%) estimates that were non-significant for all three models presented. 

For preparedness, there were 32 (10.4%) of 307 cases where the estimates were 
statistically significant only for the multilevel model. In contrast, 18 (5.9%) estimates 
were statistically non-significant only for the multilevel model. There were 11 (3.6%) 
cases in which the estimates were statistically significant for both single-level and 
multilevel models; among these 11 cases, the directions of the multilevel model 
estimates for grade eight data for Jordan and Tunisia from 2007 were —9.9 and — 
5.5, respectively. We also determined that only three (1.0%) cases were statistically 
significant for both multilevel and classroom-means models. Conversely, in 27 (8.8%) 
cases the estimates were non-significant for both multilevel and classroom-means 
models, and in six (2.0%) instances the estimates were non-significant for both the 
single-level and multilevel models. Lastly, only six (2.0%) educational systems had 
estimates that were statistically significant across all three models; the remaining 
204 (66.4%) systems had estimates that were non-significant for all three models. 

When we focused only on the multilevel model, we found that the estimates 
of 291 (19.0%) cases showed there was a significant relationship between one 
of the measures of teacher quality (either a characteristic or a behavior) and 
student performance in mathematics, and there were 1244 (81.0%) non-significant 
relationships. Among the 291 statistically significant cases, student performance 
was most commonly related to teacher experience (in 70 cases [22.8%]), followed 
by teacher education (44 cases [14.3%]), time spent on teaching mathematics (67 
cases [21.8%]), alignment (59 cases [18.9%]), and preparedness (52 cases [16.9%]). 

Finally, assuming that the results from the multilevel model are unbiased (or 
the least biased) estimates because of the substantive and empirical importance of 
student clustering within classrooms, there is a possibility that using the pooled or 
classroom-mean models leads to incorrect statistical inference. We found that there 
were 171 (11.1%) cases in which null hypotheses that coefficients were zero in 
population were incorrectly rejected (type 1 error) by either or both the single-level 
and classroom-means models. Moreover, there were 125 (8.1%) cases that failed 
to reject or incorrectly retained the null hypotheses (type 2 error). Although there 
are circumstances where classroom-means and single-level (non-clustered) analyses 
may yield substantively similar results, there were enough differences to warrant 
employing the more complex and computationally burdensome multilevel model. 
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5.3 Stability of Estimates Across Time 


An alternative way of testing the robustness of the associations between teacher 
quality measures and student outcomes is to examine the stability of estimates for 
a particular country across time. Although changes in country-level means of (for 
example) teacher experience could be explained by substantive policy or labor market 
conditions, the relationship between teacher experience and student outcomes should 
not vary dramatically over a short span of time within a particular educational system. 
To test whether this assumption holds, the variables’ coefficients estimated for each 
year were compared to establish whether there were significant differences between 
them. 

We based the theoretical framework used for the comparison procedure on the 
work of Clogg et al. (1995). They argued that, in large samples, the significance of the 
difference between the coefficients could be assessed using the following statistics: 


Bi — Ba 


Z= 
(SEB)? + (SEB? 


where Bi and po are regression coefficients from the models that are to be compared 
and SE is the associated standard error of the said estimates. The null hypothesis 
for such a test would be the equality of the coefficients. When employing a specific 
statistical test, the underlying assumptions of the test should be satisfied. Clogg et al. 
(1995) cautioned against the independence of coefficients when using this formula. 
Since these estimates were taken at different times, there is no reason to suspect 
dependency of samples and as such, this statistic is valid for the purpose at hand. 
However, since coefficients from multiple time points are being compared, the issue 
of multiple comparisons must be addressed (Curran-Everett 2000), in which the error 
rate increases as increasing numbers of pairs are compared. Several different methods 
have been suggested to adjust for this problem (Benjamini and Hochberg 1995; 
Hochberg 1988; Hommel 1988; Weisstein 2004). Here we adopted the correction 
method suggested by Benjamini and Hochberg (1995), which emphasized control 
for a false discovery rate while also minimizing the family-wise error rate. 
The variables that we assessed were: 


The number of years the teacher has been teaching (Exp) 

A teacher’s formal education to teach mathematics (Mathprep) 

Time spent on teaching mathematics (Mathtime) 

Alignment of the topics taught with national standards (Alignment) 

A teacher’s self-reported preparation to teach mathematics topics (Prepared). 


After sorting the data available from each country and each sample year, 46 
countries were found to have participated in TIMSS at least twice across the 
2003-2015 sample period at grade four, and 50 countries were found to have 
participated in TIMSS at least twice across the 2003-2015 sample period at grade 
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Table 5.2. Within-country variation across time in teacher quality coefficients 


Grade level Variable name % Countries that show significant difference (proportion 
in parenthesis) 
Grade four Alignment 13.04 ( & ) 
Mathprep 15.21 (4) 
Prepared 15.21 ( % ) 
Mathtime 15.56 (4) 
Exp 23.91 (#4) 
Grade 8 Alignment 44.00 (25) 
Mathprep 22.00 ( a ) 
Prepared 34.00 ( a ) 
(35) 


Mathtime 32.00 


wl 
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Exp 30.00 ( 
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Notes Significance level is a = 0.05. Alignment = proportion of mathematics topics reported as 
covered by teachers compared with national expectations, Mathprep = index (1-5) of teacher 
education to teach mathematics, Mathtime = mean number of minutes spent on mathematics 
teaching per week, Prepared = index (1-4) of self-efficacy to teach mathematics, Exp = experience 
teaching in years 


eight. However, not all of these countries had longitudinal data for every variable, 
so the exact number of countries differed slightly. For each variable, we determined 
the percentage and proportion of countries that had significant differences between 
their estimated coefficients at both grade four and grade eight (Table 5.2). 

At grade four, approximately 15% of countries showed a significant difference 
in estimated coefficients for most of our variables across the sampling period, with 
the exception of teacher experience. The relationship between teacher experience 
and student outcomes was found to be inconsistent in around 24% of the countries. 
Drawing on these results, one plausible hypothesis would be that there is a subset 
of countries who consistently show significant differences between the variables, 
hence the similar percentage between variables. However, although some countries 
did appear more than once, the countries who showed significant changes between 
the coefficients were randomly distributed. In other words, the temporal instability 
of relationships did not appear to be a country-specific factor. 

At grade eight, the association between teacher instructional alignment and student 
mean mathematics performance was inconsistent in almost half the sample (22 
countries), while the inconsistency for three other variables (preparedness, time 
on math, and experience) was >30%. The percentage of countries that exhibited 
significant differences was higher at grade eight than grade four. There was no clear 
pattern of countries who showed statistically significant differences in coefficients in 
all variables at grade eight. However, with such a high number of countries showing 
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statistically significant variation, countries often demonstrated significant variation 
in effect for two or more variables. 

Animportant aspect is the existence of countries who not only showed a significant 
change in the coefficient but also a change in the coefficient sign. A change in the 
sign of the trend, rather than just the amplitude, implies a higher level of coefficient 
instability. 

Both grade levels showed the existence of statistically significantly differences 
in the coefficients of all measured variables, and, notably, grade eight data showed 
much higher percentages of statistically significant differences. High percentages of 
statistically significant differences can imply an issue with the measurement tools 
or indicate a highly variant sample across time. While more research and analysis is 
needed to support these initial findings, the high percentage of statistically unstable 
coefficients restricts our ability to establish a clear pattern. The problem is further 
compounded when we consider the existence of a few differences that changed sign. 
Scale evaluation and adjustment is needed to ensure that observed variability does 
not stem from the items themselves, rather than from the research population, or as 
a result of other factors. 

This secondary analysis of the multilevel models extracted coefficients and 
compared them individually. A more holistic approach to the topic of coefficient 
stability would be to include all of those time points into a single model, thus directly 
addressing the issue of stability over time; this strategy should be considered in future 
research. 


5.4 Fixed Effect Analysis 


To further explore the relationship between teacher quality and student outcomes, 
we undertook a country-level fixed effects analysis. One of the limitations of the 
empirical approaches that we used is that they are essentially correlative, making it 
difficult to attribute causation. A more serious concern is that the models include 
a limited number of predictors, and hence are subject to unobserved variable bias. 
The models focus exclusively on measures of teacher characteristics and behavior, 
and a few student-level indicators that are available in TIMSS. The advantage of 
fixed-effects models is that analyzing changes within a given unit can provide greater 
confidence in the association between dependent and independent variables provided 
the unobserved variables are invariant. Although there is reason to doubt that this 
assumption holds fully, a fixed-effect analysis should yield somewhat more robust 
estimates than a cross-sectional regression equation. Our fixed-effect country-level 
model is restricted to only those countries that participated in the 2007, 2011, and 
2015 cycles of TIMSS for a particular grade level, and for educational systems that 
had country-level estimates available for all variables in the model. It should be 
noted that education systems are the unit of analysis here; this is not a student- 
or teacher-level fixed-effect analysis. The “effects” are therefore on the aggregate 
country level, and may not necessarily apply to particular teachers or students. 
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Table 5.3  Country-level grade four fixed-effects analysis of the relationship of teacher quality to 
student outcomes, 2007-2015 


Parameter Estimate SE p-value 
Alignment —36.89 24.04 0.13 
Mathtime 0.14 0.05 0.01 
Mathprep 18.55 9.61 0.06 
Books 55.50 20.49 0.01 
Lang 30.56 7.81 0.00 
Prepared —7.73 10.14 0.45 
Exp —2.01 1.04 0.06 
Tmale —134.89 38.29 0.00 


Notes Alignment = proportion of mathematics topics reported as covered by teachers compared 
with national expectations, Books = index (1-5) of number of books in the home, Lang = index 
(1-4) of testing language spoken in the home, Mathprep = index (1-5) of teacher education to 
teach mathematics, Mathtime = mean number of minutes spent on mathematics teaching per week, 
Prepared = index (1-4) of self-efficacy to teach mathematics, Exp = experience teaching in years, 
Tmale = index of teacher gender (female = 0, male = 1), Performance = mean student TIMSS score 
in mathematics, SE = standard error. A p-value of 0.05 or lower indicates statistical significance 


At grade four, the fixed-effects regression model did uncover a statistically 
significant association between changes in country-level means of teacher factors 
and changes in aggregate student mathematics achievement (Table 5.3). Specifically, 
countries that saw an increase in average time spent on mathematics in grade four 
saw higher mean student outcomes. Systems whose teachers had increasing levels 
of education was positive and approached statistical significance (p = 0.06), as did a 
negative relationship between changes in teacher experience and mean mathematics 
scores. Curricular alignment and teacher self-efficacy had no relationship to student 
outcomes. Interestingly, teacher gender had a strongly negative association with 
mathematics outcomes, suggesting that an increasing proportion of male teachers at 
grade four was associated with weaker mathematics scores. This is a rather curious 
result and merits more detailed investigation. 

At grade eight, time spent on mathematics was again significantly and positively 
associated with TIMSS mathematics scores, and countries whose teachers reported 
growing levels of preparedness to teach mathematics also saw higher student 
outcomes (Table 5.4). None of the other variables approached statistical significance. 
While these results should not be overstated, they do suggest that, in general, time 
spent on mathematics may have a positive relationship with student learning and that 
the failure to uncover a consistent association in other models could be due in part 
to the exclusion of relevant but unobserved variables. 

Two caveats should be kept in mind when interpreting the results of this analysis. 
As mentioned previously, the virtue of fixed-effects models is that they allow 
unobserved variables in a given unit (in this case a given educational system) to 
act as its own control, by identifying change over time. The governing assumption of 
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Table 5.4 Country-level grade eight fixed effects analysis of the relationship of teacher quality to 
student outcomes, 2007-2015 


Parameter Estimate SE p-value 
Alignment 5.82 25.02 0.82 
Mathtime 0.37 0.12 0.00 
Mathprep 16.88 6.35 0.01 
Books 23.55 17.21 0.18 
Lang 21.96 25.43 0.39 
Prepared —1.50 9.73 0.88 
Exp 1.60 1.58 0.32 
Tmale —15.79 54.29 0.77 


Notes Alignment = proportion of mathematics topics reported as covered by teachers compared 
with national expectations, Books = index (1-5) of number of books in the home, Lang = index 
(1-4) of testing language spoken in the home, Mathprep = index (1-5) of teacher education to 
teach mathematics, Mathtime = mean number of minutes spent on mathematics teaching per week, 
Prepared = index (1-4) of self-efficacy to teach mathematics, Exp = experience teaching in years, 
Tmale = index of teacher gender (female = 0, male = 1), Performance = mean student TIMSS score 
in mathematics, SE = standard error. A p-value = 0.05 or lower indicates statistical significance 


such models is that any variables that are excluded are fixed, while all the factors with 
temporal variation and likely to have a relationship with the outcome of interest are 
included. This is, of course, an optimistic assumption; there are a number of social, 
educational, economic, and policy factors that influence student achievement that 
are likely to have altered during the course of TIMSS (and perhaps even in response 
to TIMSS testing). Secondly, this analysis aggregates at the education system level, 
which reduces the number of degrees of freedom (i.e., limits sample size), and is 
something of a departure for fixed-effects studies, which more typically consider 
individuals or smaller aggregates (like school districts) rather than entire educational 
systems. The results of the analysis should therefore be treated with considerable 
caution. 


5.5 An Examination of Standard Errors 


One of the distinctive features of large-scale studies like TIMSS is the estimation 
of standard errors. As described in detail in the TIMSS user guide (Foy 2015), 
TIMSS uses a complex sampling design; in stage one a random sample of schools 
is selected, and, in stage two, a randomly-selected intact classroom is selected. If 
the study were based on a straightforward random sample of all students of a given 
age or grade level, then the estimation of standard errors could be calculated using 
conventional means. Instead, TIMSS uses a jackknifing procedure, in which schools 
are paired and then any calculations (e.g., means and regression coefficients) are 
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run separately with one of each pair weighted at zero and the weight of the other 
member of the pair doubled. Standard errors are then estimated by aggregating all 
of those separate estimates. The IEA has supported calculating means or simple 
linear regression models through the provision of macros (generated as part of the 
IEA Database Analyzer, a free specialist analysis package that can be downloaded 
from www.iea.nl/data). However, when running more complicated models (most 
especially multilevel models where students are clustered within classrooms), the 
jackknifing procedure can be computationally quite demanding, taking many hours 
of computing time to generate models for multiple countries over multiple TIMSS. 

The PISA study uses a related, but different approach in the estimation of standard 
errors. Unlike TIMSS, PISA selects a random population of 15-year-olds and then 
uses a balanced replicate weight system to calculate standard errors. However, a 
shortcut that is often used with success when analyzing PISA data is to use adjusted 
weights, such that the sum of weights equals the number of respondents in the study. 
An adjustment of this kind is necessary (if for no other reason) to prevent major 
downward bias in standard errors, since the weights in both TIMSS and PISA are 
meant to reflect the entire population of students in a country rather than the number 
of respondents (larger numbers resulting in smaller standard errors). This method 
has been used to produce results for unpublished studies that are quite similar to 
the full balanced replicate weight method, and can be convenient when conducting 
preliminary analysis (given the computational burdens of more formal procedures). 
However, it should emphasized that this strategy is not technically sound, and any 
analysis intended for publication or presentation should use the full-scale method. As 
discussed by Jerrim et al. (2017), there are published studies in reputable journals that 
have failed to use appropriate statistical procedures, and, although the substantive 
results have been quite similar, they are not considered reliable. 

We conducted a secondary analysis to determine whether the adjusted weight 
procedure yielded similar standard errors to the jackknifing procedure. All of the 
multilevel statistical models run using the jackknifing procedure were re-run with 
student weights reweighted to equal the total number of respondents. We then 
compared the standard errors of the coefficients between the adjusted weight and 
jackknifing procedures. The purpose of this analysis was to determine whether the full 
jackknifing procedure was necessary to avoid downwardly-biased standard errors. 

Although there was certainly variation in the magnitude of the differences, on 
average, we found that standard errors calculated with jackknifing procedure were 
about twice as large as those calculated using an adjusted base weight (Table 5.5). 
The difference for grade eight (2.25) was larger than for grade four (x 1.90), and 
was reasonably consistent across years. In nearly every case, the standard errors to 
account for the complex sampling design were larger with the jackknifing procedure 
than those using an adjusted base weight; in some cases, four or five times as 
large. The proportional difference was also greater for teacher-related standard errors 
(x 2.37) than for student controls (x 1.45). This confirms that it is critical to account 
for the complex sampling design in TIMSS to avoid the risk of type I errors (false 
positives). 
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5.6 Discussion 


This chapter has covered a great deal of detailed technical analysis, but the 
overarching implications are clear. First and most important, using basic statistical 
analysis based on TIMSS data, the relationship between teacher quality measures 
and student outcomes appear generally weak and inconsistent. After testing the 
robustness of the relationship between student outcomes and teacher characteristics 
in different education systems using alternative statistical models, methods of 
aggregation, and calculations of standard errors, we found that associations between 
teacher factors and student outcomes remained modest. When considered over a 
number of years, the most technically appropriate measure (multilevel estimates 
with jackknifed standard errors) suggested that there was a negligible relationship 
between teacher characteristics or teacher behaviors and student outcomes. Although 
there were education systems and years where teacher effectiveness measures were 
associated with higher student mathematics performance, this relationship was not 
robust across time and space. 

The weak associations between teacher experience and education mirror much 
of the research conducted in single-country studies. What is more surprising is that 
time on mathematics and content coverage has usually uncovered much stronger 
associations. Most studies that have found a relationship between instructional 
content and student outcomes have used measures of the volume and intensity 
of topics covered, as opposed to the alignment with national standards. This 
finding raises serious questions about the utility of standards-based reform, 
since there appears to be no strong relationship between teachers’ fidelity to 
mathematics standards and student outcomes. Whether this indicates problems with 
the measurement of teacher instructional content or the quality of the standards 
themselves deserves greater attention. 

However, our results have a number of limitations, and should not be treated 
as definitive. The models included were restricted to a fairly limited number of 
variables and student controls (most seriously, prior student performance, which is 
not available in the TIMSS dataset), raising the potential for unobserved variable 
bias. There are also potential problems with the measurements employed, including 
possible differences in interpretability across varying cultural contexts (within and 
across educational systems), the indirect measure of teacher professional knowledge 
through self-reports, and changes in the mathematics topic framework used to 
measure instructional content coverage. 
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Chapter 6 M®) 
Relationships Between Instructional creek 
Alignment, Time, Instructional Quality, 
Teacher Quality, and Student 

Mathematics Achievement 


Abstract Opportunity to learn (measured by time spent on teaching mathematics 
and mathematics content coverage) potentially acts as a mediator of instructional 
quality and teacher characteristics. Previous research used a multilevel structural 
equation model (SEM) to explore the relationship of teacher instructional quality 
(identified using a latent variable) to student mean outcomes at grade four. Here, 
this model is extended by incorporating two instructional quality variables related 
to opportunity to learn: teacher alignment with national curriculum expectations, 
and time spent on mathematics. This model was applied to grade four and grade 
eight data from the 2011 cycle of IEA’s Trends in International Mathematics 
and Science Study (TIMSS). Replicating the analysis at grade eight determined 
whether the mathematics performance of lower secondary students showed similar 
relationships with instructional quality as those observed for grade four students. 
Teacher instructional alignment and time on mathematics was found to have a weak 
mediating effect on student outcomes. 


Keywords Opportunity to learn - Structural equation models - Teacher quality - 
Trends in International Mathematics and Science Study (TIMSS) 


6.1 Introduction 


In this chapter, we address our second research question, namely what is the 
relationship between different types of teacher quality and instructional metrics 
and student achievement? Our study extends the work of Blomeke et al. (2016), 
who analyzed the TIMSS 2011 grade four data to explore the relationship between 
teacher quality variables and student outcomes. The analysis undertaken by Blomeke 
et al. (2016) incorporated similar constructs to those explored in Chap. 5 (teacher 
experience, self-efficacy, and educational preparation), and included receipt of 
professional development and teacher instructional practices. However, Blomeke 
et al. (2016) identified only a modest relationship between instructional quality and 
student mathematics achievement, and their analysis was limited to grade four data. 
In this chapter, we explore the extent to which including opportunity to learn variables 
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(time spent on teaching mathematics and content coverage) mediate the influence of 
instructional quality, using both grade four and grade eight TIMSS data. 


6.2 Data 


For the analyses in this chapter, we used the 2011 grade four and grade eight student 
data, and their mathematics teachers’ data from the IEA’s Trends in International 
Mathematics and Science Study (TIMSS). TIMSS is an international assessment 
of student mathematics and science achievement at grade four and grade eight. A 
three-stage unequal probability sampling design was used to select the sampled 
population within each education system. For TIMSS 2011 at grade four, this resulted 
in an international sample from 58 education systems comprising 297,150 grade four 
students in 14,215 classrooms in 10,422 schools, and 14,576 mathematics teachers. 
At grade eight, 50 educations systems participated in TIMSS 2011, resulting in a 
sample comprised of 287,395 grade eight students in 11,688 classrooms in 9203 
schools, and 13,190 mathematics teachers. 

Several participating education systems did not include the questions on 
mathematics standards in their country curriculum questionnaires, or the questions on 
teacher quality or instructional quality in their teacher questionnaires. We therefore 
excluded these countries entirely from our analyses, as well as any benchmarking 
participants. In many countries, several teachers taught mathematics in more than 
one classroom, and several classrooms had more than one mathematics teacher. For 
the classrooms with more than one mathematics teacher, data from one teacher was 
selected at random for the analyses. Our final sample for grade four thus included 
45 education systems comprising 233,583 grade four students nested in 11,153 
classrooms in 8268 schools, and 11,083 mathematics teachers. Our final sample for 
grade eight included 40 education systems comprising 228,107 grade eight students 
in 9002 classrooms in 7353 schools, and 8887 mathematics teachers. 

For most schools, only one classroom was selected, and thus there is no way to 
distinguish the classroom-level and school-level variances in student outcomes. We 
therefore opted to use a two-level model (i.e., students nested within classrooms) for 
each education system in this study, and did not undertake school-level analyses. 

Readers should note that, in this chapter, student gender, as well as several 
other variables, are coded differently in order to match the Blomeke et al. (2016) 
operationalization, and hence different variable names may be employed. 


6.3 Measures 


Teacher instructional alignment with national curricular expectations (Alignment), 
time spent on teaching mathematics (Mathtime), and teacher experience (Exp) were 
operationalized as described in Chap. 3. The TIMSS 2011 mathematics score was 
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used as a measure of student achievement. Several additional measures were included 
in Blomeke et al.’s (2016) grade four study and are reproduced here for both grades 
four and eight, including teacher participation in professional development and a 
latent measure of instructional quality. In order to most closely mimic the Blomeke 
et al. (2016) study, we made only minor changes to the operationalization of student 
covariates and self-reported preparation to teach mathematics. As we discuss later, 
in several instances there was fairly low internal reliability in the constructs used, 
and so we modified the model design accordingly. 


6.3.1 Professional Development 


We considered three aspects of teachers’ participation in professional development. 
The first was professional development in mathematics (PDM), which measured 
whether a teacher participated in professional development activities associated 
with mathematics content, pedagogy/instruction, and curriculum. For each activity, 
participation was coded as 1, and no participation was coded as 0. The average 
score for the three activities was used to indicate a teacher’s level of participation in 
professional development for mathematics teaching. 

The second aspect was professional development in mathematics instruction 
(PDS), which measured whether a teacher participated in professional development 
activities associated with integrating information technology into mathematics, 
mathematics assessment, and addressing individual students’ needs. Each activity 
was scored as before. The average score for these three activities was used to indicate 
a teacher’s participation in mathematics instruction professional development. 

The third aspect was professional development in collaboration (COL), assessed 
using five items: (1) “I am content with my profession as a teacher’, (2) “I am 
satisfied with being a teacher at this school”, (3) “I had more enthusiasm when I 
began teaching than I have now”, (4) “I do important work as a teacher’, and (5) “I 
plan to continue as a teacher for as long as I can’. Response options were “agree a 
lot’, “agree a little’, “disagree a little’, and “disagree a lot”; they were coded as 1, 2/3, 
1/3, and 0, respectively. The average score of these five activities was used to indicate 
a teacher’s participation in professional development in collaboration. Consistency 
coefficients (i.e., Cronbach’s alpha) varied across countries, being between 0.36 and 
0.63 for grade four mathematics teachers, and between 0.21 and 0.64 for grade eight 
mathematics teachers. The low internal consistency coefficients may suggest that the 
three aspects of professional development do not measure similar constructs. 


6.3.2. Teacher Education 


Teachers’ teaching experience was measured in years (Year). If a teacher’s highest 
education level was ISCED Level 5A or higher, their highest level of formal 
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education was coded as 1, else 0 was coded. If a teacher’s major was mathematics 
or mathematics education, their major was coded as 1, else 0 was coded. 


6.3.3. Preparation to Teach Math 


Teachers were asked to indicate “how well prepared do you feel you are to teach the 
following mathematics topics”. Response categories were “not applicable”, “very 
well prepared”, “somewhat prepared”, and “not well prepared”. If a topic was 
indicated as “not applicable”, the response was recoded as missing data, and the 
responses of “very well prepared”, “somewhat prepared” and “not well prepared” 
were coded as 1, 0.5, and 0, respectively. At grade four, we assessed this using 
eight topics related to number (NUM), seven topics related to geometric shapes and 
measures (GEO), and three topics related to data display (DAT). At grade eight, 
we assessed this using five topics related to number (NUM), five topics related to 
algebra (ALG), six topics related to geometry (GEO), and three topics related to data 
and chance (DAT). For each mathematics domain at each grade, the average score 
across the topics was used as the measure of a teacher’s preparedness to teach in that 
domain. The internal consistency coefficients varied across countries, from 0.61 to 
0.91 at grade four, and from 0.52 to 0.96 at grade eight. 


6.3.4 Instructional Quality 


We used three constructs to assess teachers’ instructional quality. Teachers were 
asked “How often do you do the following in teaching this class?” The first construct 
concerned the clarity of instruction (CI), which was measured with two items: 
(1) “summarize what students should have learned from the lesson’, and (2) “use 
questioning to elicit reasons and explanations”. The second construct concerned 
cognitive activation (CA), which was measured by: (1) “relate the lesson to students’ 
daily lives”, and (2) “bring interesting materials to class”. The third construct was 
supportive climate (SC), which was measured by: (1) “encourage all students to 
improve their performance’, and (2) “praise students for good effort’. The response 
categories were “every or almost every lesson’, “about half the lessons”, “some 
lessons”, and “never”, which were coded as 1, 2/3, 1/3, and 0, respectively. The 
average score of the items was used for each construct. The internal consistency 
coefficients of instructional quality varied across countries, from 0.20 to 0.76 at 
grade four, and from 0.21 to 0.73 at grade eight. 
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6.3.5  Student-Level Covariates 


Student gender (Gender) and number of books at home (Book) were used as 
student-level covariates. For student gender, girls were coded as 1, and boys were 
coded as 0. For the number of books at home, in this case “none or few” was coded as 
O, “one shelf’ was coded as 0.25, “one bookcase” was coded as 0.5, “two bookcases” 
was coded as 0.75, and “three or more bookcases” was coded as 1. 


6.4 Analysis 


The direct effects of instructional alignment and time on student mathematics 
achievement, and the indirect effect of instructional quality through instructional 
alignment and time were examined using multilevel structural equation modeling 
(see Fig. 6.1). 

We first examined the factor structures of professional development, preparedness 
to teach mathematics, and instructional quality at each grade, country-by-country. 
We used a multiple group approach to examine the configural invariance of the factors 
across countries, which indicated the same latent construct was represented by the 
same indicators. Next, we tested metric invariance (i.e., whether the same indicator 
showed the same factor loading on the latent construct across countries) for each 
latent construct at each grade. Note that the results are only directly comparable 
across countries when there is measurement invariance across countries. 
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Fig. 6.1 Model of hypothesized relationships at (a) grade four and (b) grade eight. Notes PD = 
professional development; PDM = professional development in mathematics, PDS = professional 
development in mathematics instruction, COL = professional development in collaboration, PRE 
= teacher’s preparedness to teach mathematics, NUM = preparedness to teach number, GEO = 
preparedness to teach shapes and measures, DAT = preparedness to teach data and chance, INQ 
= instructional quality, CI = clarity of instruction, CA = cognitive activation, SC = supportive 
climate, Major = major in mathematics, Degree = ISCED level, Exp = experience, Alignment 
= alignment with national content expectations, Mathtime = time spent on teaching mathematics, 
Gender (female = 1, male = 0), Book (1-0 index of books in the home), Performance = TIMSS 
math score 
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Second, we applied a two-level model without instructional alignment and time 
country-by-country at each grade. Student-level covariates were centered about their 
grand means. 

Finally, we constructed the hypothesized model for each grade and each country to 
examine the direct effects of instructional alignment and time, as well as the indirect 
effect of instructional quality through instructional alignment and time. The model 
was also fitted country-by-country. 

We conducted factor analyses of professional development, preparedness to teach, 
and instructional quality with teachers as the analysis units, and used teacher sampling 
weights in the analyses. For the multilevel structure equation model, the nested data 
structure (i.e., students nested within classrooms) was taken into consideration, robust 
standard errors were computed, and student and teacher sampling weights were used 
in the analyses. The model fit was evaluated using a likelihood ratio test and model 
fit indices. 


6.5 Results 


6.5.1 Descriptive Statistics 


Average mathematics achievement scores for each educational system were between 
245.94 and 606.27 at grade four, and between 332.76 and 611.10 at grade eight. 
The intraclass correlation coefficients (ICCs) for student mathematics achievement 
were between 0.07 and 0.56 at grade four, and between 0.09 and 0.79 at grade eight, 
indicating a need to explore both individual and school effects on student mathematics 
achievement. 

Instructional alignment in mathematics was between 0.34 and 0.66 at grade four, 
and between 0.26 and 0.84 at grade eight. On average, grade four mathematics 
teachers covered about 34-66% of their country grade-level mathematics standards, 
and grade eight mathematics teachers covered about 26-84% of their country 
grade-level mathematics standards in their teaching. Grade four teachers in Poland 
reported the poorest alignment with national mathematics standards, while grade 
four teacher in Australia reported the greatest percentage alignment with national 
mathematics standards. Similarly, for grade eight, mathematics teachers in Bahrain 
reported the poorest alignment with national standards, while teachers in Japan 
reported the greatest percentage alignment with national mathematics standards. 
On average, grade four teachers taught mathematics for 186-429 min per week, 
and grade eight teachers taught mathematics for 161-316 min per week. At grade 
four, mathematics teachers in Denmark spent the least time on mathematics, and 
teachers in Portugal spent most time on mathematics. At grade eight, mathematics 
teachers in Japan spent the least time, and mathematics teachers in the Lebanon 
spent most time on teaching mathematics. We noted that some countries that scored 
highly for instructional alignment did not report similarly high average instructional 
time on mathematics, and vice versa. Grade four teachers in Poland showed the 
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least alignment with national mathematics standards, but their average time spent on 
mathematics was similar to the international average. Grade eight teachers in Japan 
taught about 84% of their national grade-level mathematics standards, however, 
they spent the lowest reported time on mathematics teaching. This suggests that 
instructional alignment and time spent teaching mathematics may play different 
roles in student achievement. 


6.5.2 Measures of Professional Development, Preparedness 
to Teach, and Instructional Quality 


The factor structures of professional development, preparedness to teach 
mathematics, and instructional quality were firstly examined country-by-country 
for each grade. 

The one-factor model of professional development was built with the 
three indicators: professional development in mathematics (PDM), professional 
development in mathematics instruction (PDS), and professional development in 
collaboration (COL). This model demonstrated a convergence problem in 15 
countries (Australia, Croatia, Finland, Georgia, Germany, Honduras, Kuwait, New 
Zealand, Saudi Arabia, Slovak Republic, Slovenia, Sweden, United Arab Emirates, 
United States, and Yemen) at grade four, and in 14 countries (Australia, Chile, Ghana, 
Hong Kong, Iran, Japan, Korea, Lithuania, Morocco, Norway, Slovenia, Thailand, 
Tunisia, and Ukraine) at grade eight. These convergence problems were caused by 
the low correlation of COL with the other two indicators in these countries. Even in 
the countries with no convergence problems, the factor loadings of COL on the latent 
construct were <0.2 or even negative. Together, this indicates that COL measured a 
different construct from the latent constructs measured by PDM and PDS. To address 
the convergence problems of COL, PDM, and PDS in many countries, we used the 
three variables as separate predicators in a revised model instead of constructing a 
latent construct of professional development (Fig. 6.2). 

The one-factor model of preparedness to teach was constructed from three 
indicators at grade four: preparedness to teach number (NUM), shapes and 
measures (GEO), and data display (DAT) at grade four. At grade eight there were 
four indicators: preparedness to teach number (NUM), algebra (ALG), geometry 
(GEO) and data and chance (DAT). The model converged in all countries at 
both grades. A multiple-group approach was used to build the factor model 
for all countries at each grade. In order to assess metric invariance, the factor 
loadings of the indicators were constrained to be equal across countries. A 
likelihood ratio test (LRT) was used to examine whether the metric invariant 
model was significantly different from the base model. At grade four, we found 
that Ay* = 224.19, Adf = 88, which was statistically significant (p < 
0.001). At grade eight, we found that Ay* = 259.77, Adf = 117, which was also 
statistically significant (p <0.001). In other words, the factor loadings of the indicators 
of preparedness to teach were significantly different across countries at both grades. 
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Fig. 6.2. Model of revised relationships for (a) grade four, and (b) grade eight. Notes PD = 
professional development; PDM = professional development in mathematics, PDS = professional 
development in mathematics instruction, COL = professional development in collaboration, PRE 
= teacher’s preparedness to teach mathematics, NUM = preparedness to teach number, GEO = 
preparedness to teach shapes and measures, DAT = preparedness to teach data and chance, INQ 
= instructional quality, CI = clarity of instruction, CA = cognitive activation, SC = supportive 
climate, Major = major in mathematics, Degree = ISCED level, Exp = experience, Alignment 
= alignment with national content expectations, Mathtime = time spent on teaching mathematics, 
Gender (Female = 1, Male = 0), Book (1-0 index of books in the home), Performance = TIMSS 
math score 


The one-factor model of instructional quality was constructed from three 
indicators at both grades: clarity of instruction (CI), cognitive activation (CA), and 
supportive climate (SC). The one-factor model did not converge at grade four data 
in Northern Ireland, or at grade eight in Italy. This indicates that CI, CA, and SC 
measure different constructs in these two sets of data. Multiple-group models were 
applied to the remaining countries at each grade to examine the metric invariance. 
At grade four, we found that Ay? = 218.79, Adf = 86, which was statistically 
significant (p < 0.001). At the grade eight, we found that Ay” = 168.95, Adf = 76, 
which was statistically significant (p < 0.001). The factor loadings of the indicators 
of instructional quality were significantly different across countries at both grades. 


6.5.3 Effect of Teacher Quality and Instructional Quality 
on Student Mathematics Achievement 


As the factor analyses suggested that PDM, PDS, and COL measured different latent 
constructs in many countries, the three indicators were used directly as predictors in 
the models. As the factor loadings of preparedness to teach and instructional quality 
differed across countries, the models to explore the effects of teacher quality and 
instructional quality were built country-by-country rather than in a multiple-group 
fashion. 

The model did not converge for the grade four data from Australia, Malta, 
Netherlands, Romania, or England, or for the grade eight data from the Syrian 
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Arab Republic or Tunisia! (Tables 6.1 and 6.2). In most countries, the chi-square 
statistics were statistically significant, which is common with large sample sizes. 
In countries with no convergence problems, Root mean square error approximations 
(RMSEAs) were generally <0.02, standardized root mean square residuals (SRMRs) 
for the within and between levels were <0.08, and the comparative fit index (CFI) 
and Tucker Lewis index (TLI) were >0.80. 

In many countries there were no convergence problems shown by the standardized 
coefficients of instructional quality and teacher quality effects on student mathematics 
achievement (Tables 6.3 and 6.4). At grade four, at least one of the professional 
development indicators showed significant effects on instructional quality in 16 
countries. At grade eight, at least one indicator of professional development showed 
a significant and positive effect on instructional quality in 24 out of the 38 countries. 
In these countries, the effects of professional development indicators varied from 0.2 
to 0.4. A teacher’s score for professional development activities in mathematics, 
mathematics instruction, and collaboration was directly and positively linked to 
instructional quality, with higher scores indicating better instructional quality. 
However, these three professional development indicators showed weak relationships 
with student mathematics achievement in both grades four and eight. Their direct 
effects on student mathematics achievement were significantly positive in only eight 
out of 40 countries at grade four, and seven out of 38 countries at grade eight. In 
most countries, teachers’ participation in professional development activities did not 
have any significant direct effects on student mathematics achievement. 

The effects of preparedness to teach on instructional quality were significant at 
grade four in 15 countries, and at grade eight in 14 countries. The effect sizes of 
preparedness to teach ranged between 0.2 and 0.5. The better prepared teachers felt 
to teach mathematics topics, the better their instructional quality. The direct effects 
of preparedness to teach on student mathematics achievement were non-significant 
in most countries. The direct effects of preparedness to teach were significant and 
positive in only two out of 40 countries at grade four, and nine out of 38 countries at 
grade eight. 

The three teacher education background indicators, experience, degree, and major, 
affected instructional quality significantly and positively in nine out of 40 countries 
at grade four, and in three out of 38 countries at grade eight. In comparison, their 
direct effects on student mathematics achievement were significant in more countries. 
Their effects were positively significant in 13 countries at grade four, and in 14 
countries at grade eight. In these countries, the teachers with more experience, a 
higher degree, and who majored in mathematics major had more positive effects on 
student mathematics achievement. However, teachers’ experience, degree, and major 
did not have any significant impact on their instructional qualities in many countries. 

The direct effects of instructional quality on student mathematic achievement were 
significant and positive in only two out of 40 countries at grade four, and in seven 
out of 38 countries at grade eight. For the non-significant effects of instructional 


‘As the variance of teacher major in the available grade eight data from Romania was zero, major 
was not used in Romania’s grade eight model. 
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Table 6.1 Model fit of the grade four country-by-country models of the effects of teacher and 
instructional quality 


Education x2 df Pp RMSEA | CFI TLI SRMR_W_ | SRMR_B 
system 

Bahrain 76.92 | 42 0.00 0.02 0.87 0.79 0.00 0.07 
Botswana 96.50 | 42 0.00 0.02 0.84 0.75 0.00 0.08 
Chile 52.92 | 42 0.12 0.01 0.92 0.87 0.00 0.05 
Chinese 51.71 | 42 0.14 0.01 0.98 0.98 0.00 0.06 
Taipei 

Croatia 52.42 | 42 0.13 0.01 0.97 0.96 0.00 0.06 
Czech 52.43 | 42 0.13 0.01 0.98 0.97 0.00 0.05 
Republic 

Denmark 73.65 | 42 0.00 0.02 0.91 0.86 0.00 0.06 
Finland 72.83 | 42 0.00 0.01 0.93 0.90 0.00 0.06 
Georgia 91.42 | 42 0.00 0.02 0.81 0.70 0.00 0.06 
Germany 73.45 | 42 0.00 0.02 0.94 0.90 0.00 0.06 
Honduras 70.39 | 42 0.00 0.02 0.83 0.73 0.00 0.08 
Hong Kong 86.20 | 42 0.00 0.02 0.87 0.80 0.00 0.08 
Hungary 84.49 | 42 0.00 0.01 0.93 0.90 0.00 0.06 
Tran 37.89 | 42 0.65 0.00 1.00 1.02 0.00 0.03 
Ireland 99.08 | 42 0.00 0.02 0.86 0.78 0.00 0.06 
Italy 52.72 | 42 0.12 0.01 0.96 0.94 0.00 0.05 
Japan 52.61 | 42 0.13 0.01 0.98 0.97 0.00 0.07 
Korea 45.37 | 42 0.33 0.00 1.00 0.99 0.00 0.05 
Kuwait 37.98 | 42 0.65 0.00 1.00 1.05 0.00 0.03 
Lithuania 86.26 | 42 0.00 0.02 0.94 0.90 0.00 0.05 
Morocco 77.32 | 42 0.00 0.02 0.87 0.80 0.00 0.06 
Oman 44.32 | 42 0.37 0.00 0.99 0.99 0.00 0.05 
New 82.44 | 42 0.00 0.01 0.91 0.85 0.00 0.05 
Zealand 

Northern 71.67 | 42 0.00 0.02 0.97 0.95 0.00 0.05 
Ireland 

Norway 62.20 | 42 0.02 0.01 0.94 0.91 0.00 0.07 
Poland 104.41 | 42 0.00 0.02 0.87 0.80 0.00 0.07 
Portugal 65.17 | 42 0.01 0.01 0.93 0.89 0.00 0.05 
Qatar T9.AS | 42 0.00 0.02 0.91 0.86 0.00 0.05 
Saudi 38.87 | 42 0.61 0.00 1.00 1.02 0.00 0.04 
Arabia 

Serbia 44.94 | 42 0.35 0.00 0.99 0.98 0.00 0.06 
Singapore 80.25 | 42 0.00 0.01 0.91 0.87 0.00 0.05 


(continued) 
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Table 6.1 (continued) 


Education ¢ df Dp RMSEA | CFI TLI SRMR_W |SRMR_B 
system 

Slovak 56.65 | 42 0.07 0.01 0.98 0.97 0.00 0.04 
Republic 

Slovenia 69.49 | 42 0.00 0.01 0.95 0.92 0.00 0.05 
Spain 63.82 | 42 0.02 0.01 0.95 0.92 0.00 0.05 
Sweden 55.52 | 42 0.08 0.01 0.97 0.95 0.00 0.06 
Thailand 64.46 | 42 0.01 0.01 0.93 0.88 0.00 0.09 
Tunisia 52.50 | 42 0.13 0.01 0.96 0.94 0.00 0.06 
United Arab | 65.27 | 42 0.01 0.01 0.94 0.91 0.00 0.03 
Emirates 

United 48.04 | 42 0.24 0.00 0.99 0.99 0.00 0.03 
States 

Yemen 36.03 | 42 0.73 0.00 1.00 1.22 0.00 0.05 


Notes x? = chi-square, df = degrees of freedom, p = p-value, RMSEA = root mean square 
error of approximation, CFI = comparative fit index, TLI = Tucker-Lewis index, SRMR_W = 
standardized root mean square residual for the within level, and SRMR_B = standardized root 
mean square residual for the between level 


quality on student mathematics achievement, the indirect effects of teacher quality 
indicators through instructional quality were non-significant in most countries. The 
significant indirect effects were found in only two countries at grade four, and in 
five countries at grade eight. In Korea, professional development in collaboration 
and preparedness to teach showed significant and positive indirect effects through 
instructional quality on student mathematics achievement at grade eight. When the 
number of professional development activities in mathematics instruction undertaken 
by a grade four teacher in Lithuania increased, their instructional quality and average 
student mathematics achievement level in their classroom also increased. In Oman, 
the effects of professional development in mathematics and degree were mediated 
by instructional quality at grade four, and the effects of professional development 
in mathematics instruction, preparedness to teach, and degree were mediated by 
instructional quality at grade eight. At grade eight, the effects of professional 
development in collaboration were mediated by instructional quality in Romania, 
the effects of professional development in mathematics instruction were mediated 
by instructional quality in Saudi Arabia, and the effects of preparedness to teach 
were mediated by instructional quality in Macedonia. 
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Table 6.2. Model fit of the grade eight country-by-country models of the effects of teacher and 
instructional quality 


Education x2 df Pp RMSEA | CFI TLI SRMR_W |SRMR_B 
system 

Australia 96.55 | 54 0.00 0.01 0.94 0.91 0.00 0.07 
Bahrain 156.51 | 54 0.00 0.02 0.90 0.86 0.00 0.08 
Botswana 66.13 | 54 0.12 0.01 0.96 0.94 0.00 0.05 
Chile 91.77 | 54 0.00 0.01 0.96 0.94 0.00 0.06 
Chinese 68.98 | 54 0.08 0.01 0.98 0.97 0.00 0.06 
Taipei 

England 69.88 | 54 0.07 0.01 0.99 0.99 0.00 0.05 
Finland 102.86 | 54 0.00 0.02 0.93 0.90 0.00 0.07 
Georgia 201.37 | 54 0.00 0.03 0.73 0.61 0.00 0.07 
Ghana 66.55 | 54 0.12 0.01 0.97 0.95 0.00 0.06 
Honduras 67.36 | 54 0.10 0.01 0.97 0.95 0.00 0.07 
Hong Kong 73.81 | 54 0.04 0.01 0.95 0.93 0.00 0.06 
Hungary 67.84 | 54 0.10 0.01 0.99 0.98 0.00 0.05 
Indonesia 87.88 | 54 0.00 0.01 0.91 0.87 0.00 0.09 
Iran 79.54 | 54 0.01 0.01 0.95 0.93 0.00 0.06 
Israel 169.05 | 54 0.00 0.02 0.86 0.80 0.00 0.08 
Italy 111.92 | 54 0.00 0.02 0.93 0.89 0.00 0.07 
Japan 79.59 | 54 0.01 0.01 0.96 0.94 0.00 0.07 
Jordan 68.33 | 54 0.09 0.01 0.95 0.93 0.00 0.06 
Korea 74.60 | 54 0.03 0.01 0.98 0.97 0.00 0.06 
Lebanon 81.72 | 54 0.01 0.01 0.93 0.90 0.00 0.07 
Lithuania 81.98 | 54 0.01 0.01 0.94 0.91 0.00 0.05 
Macedonia 92.65 | 54 0.00 0.02 0.87 0.81 0.00 0.06 
Malaysia 104.28 | 54 0.00 0.01 0.90 0.85 0.00 0.06 
Morocco 77.30 | 54 0.02 0.01 0.93 0.91 0.00 0.04 
Oman 69.63 | 54 0.07 0.01 0.98 0.97 0.00 0.04 
New 95.42 | 54 0.00 0.01 0.95 0.92 0.00 0.06 
Zealand 

Norway 80.65 | 54 0.01 0.01 0.96 0.95 0.00 0.07 
Romania 78.76 | 48 0.00 0.01 0.95 0.93 0.00 0.05 
Palestinian 92.44 | 54 0.00 0.01 0.89 0.85 0.00 0.06 
Authority 

Saudi 53.18 | 54 0.51 0.00 1.00 1.01 0.00 0.05 
Arabia 

Singapore 103.45 | 54 0.00 0.01 0.92 0.89 0.00 0.06 
Slovenia 83.74 | 54 0.01 0.01 0.97 0.96 0.00 0.05 


(continued) 
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Table 6.2 (continued) 


Education ta df Dp RMSEA | CFI TLI SRMR_W |SRMR_B 
system 

South 124.40 | 54 0.00 0.01 0.87 0.82 0.00 0.08 
Africa 

Sweden 72.42 | 54 0.05 0.01 0.97 0.95 0.00 0.06 
Thailand 64.12 | 54 0.16 0.01 0.97 0.95 0.00 0.05 
Turkey 60.83 | 54 0.24 0.00 0.99 0.99 0.00 0.05 
Ukraine 154.77 | 54 0.00 0.02 0.87 0.82 0.00 0.08 
United 144.12 | 54 0.00 0.01 0.90 0.85 0.00 0.07 
States 


Notes y7 = chi-square, df = degrees of freedom, p = p-value, RMSEA = root mean square 
error of approximation, CFI = comparative fit index, TLI = Tucker-Lewis index, SRMR_W = 
standardized root mean square residual for the within level, and SRMR_B = standardized root 
mean square residual for the between level 


6.5.4 Effect of Instructional Alignment and Teaching Time 
on Student Mathematics Achievement 


Based on the models outlined in Sects. 6.3 and 6.4, the final model was built on 
instructional alignment and class time spent on teaching mathematics. For each 
country we examined both the direct and mediating effects of instructional alignment 
and time spent on teaching mathematics on student mathematics achievement. These 
models were built for countries where no convergence problems were identified in 
the previous models. All models converged (Tables 6.5 and 6.6). As in previous 
models, the chi-square statistics were statistically significant in the final model 
in most countries. In countries with no convergence problems, RMSEAs were 
generally <0.02, SRMRs for the within and between levels were <0.08, and CFI and 
TLI were >0.80. However, we found that the model did not have a good fit to the 
grade eight data for Georgia. 

The effects of instructional alignment and time spent teaching mathematics 
on student achievement were examined after controlling for teacher and student 
characteristics (Tables 6.7 and 6.8). Teachers’ instructional quality showed significant 
and positive effects on teachers’ instructional alignment at grade four in Croatia, 
Iran, New Zealand, and Yemen, and at grade eight in Indonesia, Saudi Arabia, 
Ukraine, and the United States. In these countries, teachers with higher instructional 
quality levels also reported better alignment with national mathematics standards. 
Teachers’ instructional alignment showed significantly positive effects on student 
mathematics at grade four in Denmark, Georgia, and Germany, and at grade eight in 
England, Indonesia, Italy, Lebanon, New Zealand, and Norway. Greater alignment 
between teachers’ instruction and national mathematics standards was directly and 
positively associated with higher student achievement scores. Taking the effects of 
instructional quality on instructional alignment and the effects of instructional quality 
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Table 6.5 Model fit of the grade four country-by-country models of the effects of instructional 
alignment and time on task 


Education x2 df Pp RMSEA | CFI TLI SRMR_W |SRMR_B 
system 

Bahrain 109.80 | 65 0.00 0.01 0.84 0.78 0.00 0.07 
Botswana 105.58 | 65 0.00 0.02 0.87 0.82 0.00 0.07 
Chile 76.18 | 65 0.16 0.01 0.92 0.89 0.00 0.06 
Chinese 82.77 | 65 0.07 0.01 0.97 0.96 0.00 0.07 
Taipei 

Croatia 74.44 | 65 0.20 0.01 0.98 0.96 0.00 0.06 
Czech 113.96 | 65 0.00 0.01 0.93 0.90 0.00 0.06 
Republic 

Denmark 105.29 | 65 0.00 0.02 0.89 0.84 0.00 0.06 
Finland 120.15 | 65 0.00 0.01 0.89 0.85 0.00 0.06 
Georgia 114.11 | 65 0.00 0.01 0.82 0.74 0.00 0.06 
Germany 95.12 | 65 0.01 0.01 0.94 0.91 0.00 0.06 
Honduras 99.41 | 65 0.00 0.01 0.80 0.71 0.00 0.09 
Hong Kong | 129.40 | 65 0.00 0.02 0.83 0.76 0.00 0.08 
Hungary 100.57 | 65 0.00 0.01 0.95 0.92 0.00 0.06 
Iran 72.02 | 65 0.26 0.01 0.98 0.97 0.00 0.04 
Ireland 290.45 | 65 0.00 0.03 0.48 0.27 0.00 0.06 
Italy 83.42 | 65 0.06 0.01 0.93 0.90 0.00 0.06 
Japan 78.48 | 65 0.12 0.01 0.98 0.97 0.00 0.07 
Korea 72.19 | 65 0.25 0.01 0.99 0.99 0.00 0.06 
Kuwait 52.07 | 65 0.88 0.00 1.00 1.15 0.00 0.04 
Lithuania 108.83 | 65 0.00 0.01 0.93 0.91 0.00 0.05 
Morocco 118.22 | 65 0.00 0.01 0.82 0.74 0.00 0.08 
Oman 79.41 | 65 0.11 0.01 0.96 0.95 0.00 0.05 
New 117.87 | 65 0.00 0.01 0.89 0.84 0.00 0.05 
Zealand 

Northern 102.28 | 65 0.00 0.01 0.96 0.94 0.00 0.06 
Ireland 

Norway 109.55 | 65 0.00 0.02 0.88 0.83 0.00 0.07 
Poland 156.98 | 65 0.00 0.02 0.82 0.75 0.00 0.06 
Portugal 87.78 | 65 0.03 0.01 0.93 0.90 0.00 0.05 
Qatar 115.36 | 65 0.00 0.02 0.89 0.84 0.00 0.06 
Saudi 71.87 | 65 0.13 0.01 0.96 0.95 0.00 0.05 
Arabia 

Serbia 96.18 | 65 0.01 0.01 0.89 0.85 0.00 0.07 
Singapore 99.78 | 65 0.00 0.01 0.93 0.90 0.00 0.05 


(continued) 
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Table 6.5 (continued) 


Education ¢ df Dp RMSEA | CFI TLI SRMR_W |SRMR_B 
system 

Slovak 97.25 | 65 0.01 0.01 0.96 0.94 0.00 0.04 
Republic 

Slovenia 95.56 | 65 0.01 0.01 0.94 0.91 0.00 0.06 
Spain 95.44 | 65 0.01 0.01 0.93 0.90 0.00 0.06 
Sweden 108.15 | 65 0.00 0.01 0.91 0.87 0.00 0.07 
Thailand 107.45 | 65 0.00 0.01 0.87 0.82 0.00 0.09 
Tunisia 80.84 | 65 0.09 0.01 0.94 0.92 0.00 0.07 
United Arab | 121.14 | 65 0.00 0.01 0.88 0.83 0.00 0.04 
Emirates 

United 89.06 | 65 0.03 0.01 0.97 0.96 0.00 0.04 
States 

Yemen 71.66 | 65 0.27 0.00 0.89 0.85 0.00 0.06 


Notes yx? = chi-square, df = degrees of freedom, p = p-value, RMSEA = root mean square 
error of approximation, CFI = comparative fit index, TLI = Tucker-Lewis index, SRMR_W = 
standardized root mean square residual for the within level, and SRMR_B = standardized root 
mean square residual for the between level 


on student mathematics achievement together, instructional alignment showed a 
significant mediating effect on the relationship between instructional quality and 
student mathematics achievement only at grade eight in Indonesia. 

Teachers’ instructional quality showed significant and positive effects on teachers’ 
instructional time on mathematics at grade four in Croatia, Hungary, New Zealand, 
and Saudi Arabia, and at grade eight in Palestinian and Thailand. The higher the 
instructional quality teachers had, the more time they would spend on mathematics. 
Teachers’ instructional time on mathematics showed significantly positive effects 
on student mathematics outcomes at grade four in Bahrain, Iran, and United Arab 
Emirates, and at grade eight in Chile, Chinese Taipei, Japan, Jordan, Morocco, New 
Zealand, Thailand, and Ukraine. The more time teachers spent on mathematics, 
the better mathematics achievement their students would have. Taking the effects 
of instructional quality and instructional time together, instructional time on 
mathematics showed a significant mediating effect on the relationship between 
instructional quality and student mathematics achievement in the grade eight data in 
Thailand. 


6.6 Discussion 


In the original Blomeke et al. (2016) study, the researchers found that, although the 
latent construct of instructional quality was influenced by professional development, 
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Table 6.6 Model fit of the grade eight country-by-country models of the effects of instructional 


alignment and time on task 


Education X df Pp RMSEA | CFI TLI SRMR_W_ | SRMR_B 
system 

Australia 138.73 | 79 0.00 0.01 0.92 0.89 0.00 0.07 
Bahrain 268.31 | 79 0.00 0.02 0.82 0.76 0.00 0.09 
Botswana 113.22 | 79 0.01 0.01 0.88 0.84 0.00 0.07 
Chile 141.37 | 79 0.00 0.01 0.93 0.91 0.00 0.07 
Chinese Taipei 96.74 | 79 0.09 0.01 0.98 0.97 0.00 0.07 
England 121.87 | 79 0.00 0.01 0.97 0.96 0.00 0.06 
Finland 131.26 | 79 0.00 0.01 0.93 0.90 | 0.00 0.07 
Georgia 638.05 | 79 0.00 0.04 0.14 —0.17 0.00 0.09 
Ghana 90.96 | 79 0.17 0.01 0.97 0.96 0.00 0.06 
Honduras 101.16 | 79 0.05 0.01 0.95 0.93 0.00 0.08 
Hong Kong 96.59 | 79 0.09 0.01 0.96 0.94 | 0.00 0.07 
Hungary 106.87 | 79 0.02 0.01 0.98 0.97 0.00 0.05 
Indonesia 123.26 | 79 0.00 0.01 0.89 0.85 0.00 0.09 
Iran 120.63 | 79 0.00 0.01 0.92 0.90 | 0.00 0.06 
Israel 178.35 | 79 0.00 0.02 0.87 0.83 0.00 0.07 
Italy 140.08 | 79 0.00 0.02 0.92 0.90 | 0.00 0.07 
Japan 118.03 | 79 0.00 0.01 0.94 0.92 0.00 0.07 
Jordan 92.42 | 79 0.14 0.01 0.96 0.94 | 0.00 0.06 
Korea 102.24 | 79 0.04 0.01 0.98 0.97 0.00 0.06 
Lebanon 123.15 | 79 0.00 0.01 0.90 0.86 0.00 0.07 
Lithuania 141.28 | 79 0.00 0.01 0.88 0.83 0.00 0.06 
Macedonia 110.88 | 79 0.01 0.01 0.89 0.86 0.00 0.06 
Malaysia 148.85 | 79 0.00 0.01 0.87 0.82 0.00 0.07 
Morocco 125.70 | 79 0.00 0.01 0.89 0.85 0.00 0.05 
Oman 100.25 | 79 0.05 0.01 0.97 0.96 0.00 0.05 
New Zealand 143.51 | 79 0.00 0.01 0.92 0.89 0.00 0.07 
Norway 111.55 | 79 0.01 0.01 0.96 0.95 0.00 0.07 
Palestinian 103.43 | 79 0.03 0.01 0.93 0.91 0.00 0.06 
Authority 

Romania 136.62 | 71 0.00 0.01 0.91 0.87 0.00 0.06 
Saudi Arabia 86.53. | 79 0.26 0.01 0.97 0.96 0.00 0.06 
Singapore 131.52 | 79 0.00 0.01 0.93 0.90 | 0.00 0.06 
Slovenia 111.16 | 79 0.01 0.01 0.97 0.96 0.00 0.05 


(continued) 
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Table 6.6 (continued) 


Education 5 ta df p RMSEA | CFI TLI SRMR_W | SRMR_B 
system 
South Africa 156.46 | 79 0.00 0.01 0.86 0.81 0.00 0.08 
Sweden 96.00 | 79 0.09 0.01 0.97 0.96 0.00 0.06 
Thailand 104.62 | 79 0.03 0.01 0.93 0.90 | 0.00 0.06 
Turkey 90.41 | 79 0.18 0.01 0.99 0.99 0.00 0.05 
Ukraine 213.55 | 79 0.00 0.02 0.84 0.78 0.00 0.09 
United States 200.53 | 79 0.00 0.01 0.88 0.83 0.00 0.07 
Notes x? = chi-square, df = degrees of freedom, p = p-value, RMSEA = root mean square error of approximation, 


CFI = comparative fit index, TLI = Tucker-Lewis index, SRMR_W = standardized root mean square residual 
for the within level, and SRMR_B = standardized root mean square residual for the between level 


experience, and sense of preparedness, instructional quality was only weakly related 
to student outcomes in grade four. We adapted the model developed by Blomeke et al. 
(2016) and applied it to the same cycle of TIMSS data (2011) to explore in greater 
detail the direct, indirect, and mediating effects of teacher effectiveness measures. 
The aim was to test whether the inclusion of opportunity to learn variables would 
strengthen the overall model and influence the statistical impact of instructional 
quality. The direct effects of instructional alignment and time on student mathematics 
achievement, and their mediating effects on the relationship between instructional 
quality and student mathematics were found to be positive and significant only in 
a small number of countries. As noted in Chap. 5, instructional alignment and time 
spent on teaching mathematics had a limited and inconsistent relationship to student 
outcomes, even including additional teacher-related variables, such as receiving 
professional development and the latent construct of instructional quality, which 
we incorporate and understand in our model as identifying pedagogical quality, as 
distinct from opportunity to learn variables. 

The findings here are consistent with those in Chap. 5, but are in stark contrast 
with the results of previous analyses using PISA data (see Jerrim et al. 2017; Schmidt 
et al. 2015). These disparate outcomes could be the result of the different design 
of the two studies. TIMSS selects intact classrooms, whereas the PISA samples 
15-year-olds in different classrooms and at different grades. As a result it is difficult 
to distinguish school or within-school/between-classroom effects in TIMSS studies. 
In addition, the measure of content coverage is quite different between the two sets 
of studies: the Schmidt et al. (2015) work (using PISA) measured curricular intensity 
as opposed to alignment with national standards (the TIMSS measure). Further, the 
measure of opportunity to learn in TIMSS is teacher reported, while the comparable 
PISA variable is student reported. All of these factors could account for the differing 
ability of the present analysis to replicate the results from PISA studies, but in doing 
so they raise the possibility that the impact of time spent on teaching mathematics 
or instructional content are influenced by study design. 
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A final note relates to the issue of international comparability. The varying 
relationships among indicators of teacher quality across educational systems 
(measured by factor loadings and structural equation model structure) reinforces 
the challenge of merging data from multiple countries into a general global model of 
teacher effects on student learning. Our results suggest that the interrelationship of 
teacher factors to one another, and to student mathematics learning, is conditioned 
by cultural and national policy contexts, and that additional measures need to be 
included to identify the source of these differences. 
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Chapter 7 ®) 
Teacher Effectiveness and Educational ectics 
Equity 


Abstract Inequalities in teacher quality may be examined using several different 
lenses. To better investigate the relationships between student equity and teacher 
quality and instructional metrics, educational inequality in student performance 
across countries and time was explored as (1) variation in student mathematics 
scores, and (2) differences in socioeconomic status using descriptive, regression, and 
fixed-effects techniques. Measures of teacher quality (as measured by experience, 
education, preparedness, time spent on teaching mathematics, and instructional 
alignment) appeared to have only a limited effect on aggregate or within-classroom 
variation in student outcomes. The results also suggest that teacher quality may be 
more equitable at grade four than at grade eight, as measured by differences between 
higher and lower socioeconomic status classrooms. 


Keywords Educational inequality - Teacher quality - Trends in International 
Mathematics and Science Study (TIMSS) 


7.1 Inequality in Teacher Quality: The Conceptual Terrain 


Our study has focused on the relationship between teacher factors and mean student 
achievement. However, average performance can conceal massive differences among 
different groups of students. Hypothetically two countries can have very similar 
mean achievement but dramatically different distributions in achievement. This is 
the issue of educational equity, which has become a major focus of policymakers 
and researchers since at least the 1960s. In fact, a persuasive argument can be made 
that educational equity is as important as mean achievement. 

Concerns about equity are grounded in two issues, one practical and the other 
normative. First, despite the argument that there is an equity-efficiency trade-off (that 
overall increases in student learning come at the cost of more uneven distribution 
of equity in education) recent evidence suggests that no such trade-off exists, and, 
in reality, that greater educational equity is associated with higher average student 
performance (Parker et al. 2018) As aconsequence, educational systems that generate 
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more unequal outcomes may be depressing their stock of human capital by failing 
to tap into the potential of all of their students, with deleterious consequences for 
national prosperity. 

Second, educational inequality is also intrinsically problematic. The implicit 
social contract in most modern societies is that unequal rewards in the marketplace 
(i.e., large differences in wealth and income) can only be justified on the basis of 
fair competition. Educational systems have traditionally been viewed as the key 
mechanism for establishing this condition, by giving all students a fair chance to 
develop their talents. If some students are systematically disadvantaged in their 
chance to earn a good education, it calls into question the legitimacy of the social 
order. This is particularly so when there are entire groups of children that are 
systematically disadvantaged based on their background circumstances, such as their 
gender, race and ethnicity, socioeconomic status, or place of national origin, to name 
just a few examples. 

In outlining these conceptual issues, we have thus far glided over a very important 
distinction between inequality in educational outcomes and inequality in educational 
opportunities. While differences in educational outcomes may be strongly suggestive 
of background unfairness, and very high variation in student performance may signify 
a failure to maximize educational potential, differences in educational opportunities 
are more morally suspect and point to possible causes of educational inequality. It 
is patently unfair if some children are short-changed solely due to their ascriptive 
characteristics (gender, poverty, etc.), especially when those disadvantages are the 
product of policy. When schools are structured in such a way to ensure that more 
advantaged students have access to, for example, a more rigorous curriculum, higher 
quality teachers, or better facilities, then the educational system, and the people 
that manage and support it, are culpable for inequality. However, because policies 
are malleable, the extent to which policy is responsible for unequal opportunities 
indicates that these inequalities are also malleable. Policies can be changed. 

Although most studies of educational inequality have focused on specific 
countries, international and comparative studies are extremely valuable. The specific 
cultural and institutional contexts may influence the kinds of inequalities that 
manifest in particular countries, and so require careful examination on their own 
terms. However, there are some inequalities that are extremely common across 
educational systems, and these differences can provide important lessons about what 
causes inequality and how to reduce it. 

Arguably the most universal educational inequality is a consequence of 
socioeconomic status (SES). Although other types of inequality are certainly 
important, in virtually every educational system, students whose parents have lower 
incomes and less formal education perform worse by virtually any educational 
metric. Whether using PISA or TIMSS data, international large-scale assessments 
indicate that low-SES students register lower mean scores than their more affluent 
peers (Chudgar and Luschei 2009; Montt 2011; Schmidt et al. 2015). The precise 
nature of this relationship remains in dispute. While there is considerable evidence 
that low-SES children typically have fewer opportunities and resources in their 
homes and communities, the role of in-school factors remains unclear, and may vary 
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greatly across educational systems. For example, research based on the United States 
indicates that high-poverty students usually have lower-quality teachers, whether 
measured by experience, educational background, or more sophisticated value-added 
modeling (Goldhaber et al. 2015). Results from the OECD’s Teaching and Learning 
International Survey (TALIS) similarly show lower levels of teacher professionalism 
in economically disadvantaged schools in multiple countries. (OECD 2016) But a 
group of studies (Akiba et al. 2007; Burroughs and Chudgar 2017; Chudgar and 
Luschei 2009) have found that, by some metrics, there are countries where more 
economically disadvantaged students have access to higher quality teachers. There 
are other in-school factors where the inequalities are more stark and consistent, 
however. Comparative analysis by Chmielewski (2014) and Schmidt et al. (2015) 
using PISA data, and Schmidt et al. (2001) using TIMSS data, indicate persistent 
inequalities in opportunity to learn rigorous mathematics content. 


7.2 A Comparative Analysis of Inequality in Teacher 
Effectiveness 


In this chapter, the basic approach is similar to that used in Chap. 5, except that, 
instead of treating mean student performance as the dependent variable, our focus is 
on educational inequality. Whereas Chap. 5 suggested that there was a fairly weak and 
inconsistent relationship between teacher quality measures and student outcomes, 
here we explore whether teachers’ characteristics and behavior, as measured by 
TIMSS items, are related to educational inequality, and consequently whether 
changes in teacher quality have a role in promoting greater educational equity. As 
in Chap. 5, we aimed to identify common patterns across time and space, with an 
emphasis on consistent relationships, but, as discussed in Chap. 5, there are a number 
of methodological and substantive limitations to this approach, so the results should 
be treated as preliminary. 

We examined two measures of inequality: variation in student performance and 
differences between high- and low-SES classrooms. In our first set of analyses, 
we followed Montt (2011) and Mullis et al. (2016) in assessing overall inequality 
by using standard deviations in student outcomes as our measure. This measure 
of inequality captures overall differences in student outcomes without focusing 
on subgroup differences. More compressed distributions in TIMSS mathematics 
performance are considered as indicating lower levels of inequality in outcomes. As 
with the analyses of average outcomes, we focused on the 2003—2015 cycles of 
TIMSS, since many of the variables of interest were absent from the 1995 and 1999 
iterations. 
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7.2.1 Inequality as Within-Country Variation I: Descriptives 


The first step is to examine mean differences in within-country standard deviations, 
ignoring classroom-level effects. Country-level analysis was conducted for each 
country participating in TIMSS between 2003 and 2015 for both grade four and 
grade eight (Tables 7.1 and 7.2). At grade four, within-country score variation across 
all cycles ranged from a high of 114 points for Yemen in 2003, to a low of 53 points 
for the Netherlands in 2011. At grade eight, the highest standard deviation across 
all cycles considered ranged from a high of 113 points for Saudi Arabia in 2015, to 
a low of 58 points for Australia in 2011. At both grades four and eight, there was 
a general tendency toward greater within-country variation in student mathematics 
test scores in the Middle Eastern/Arab-speaking countries. 

Delving deeper into the data, we examined the subset of countries that participated 
in TIMSS between 2007 and 2015: there were 22 countries that participated in all 
cycles of TIMSS over this period at grade four, and 25 countries at grade eight. 
At grade four, the average within-country variation in mathematics scores changed 
very little overall, being 79.4 in 2007, and 80.0 in both 2011 and 2015. There was 
a fair degree of movement for particular countries, however. An equal number of 
educational systems (11 each) witnessed declines and increases in the size of standard 
deviations. The largest increases in inequality were exhibited by Iran (a 17 point 
increase) and the United States (a six point increase), while the largest declines were 
in Japan (seven points) and the Slovak Republic (five points). 

Patterns differed for grade eight. Most especially, there was a great deal more 
variation in the size of within-country performance variation. The standard deviations 
across the 25 countries were 85 points in 2007 and 2015, and 80 points in 2011. 
Further, the magnitude of the changes was far greater than in grade four. The average 
increase for countries that saw an increase in inequality was 13 points (compared to 
only four points at grade four). Similarly, the average size of the decline in those that 
saw shrinking standard deviations was 10 points at grade eight, compared with only 
three points at grade four. On balance, there were more countries with a shrinking 
inequality score (15 systems) than countries with a growing inequality score (10 
systems). It is notable that, in 2015, the United States saw larger within-country 
variation in mathematics outcomes than in 2007 at both grade levels. 

Examination of within-country trends reveals few clear patterns. Concentrating 
on those systems that participated in at least three of the last four cycles of TIMSS, 
the data indicate no consistent trends at grade four. At grade eight, there was a steady 
increase in standard deviations between 2003 and 2015 in two systems (Armenia, 
totaling five points and Palestine, seven points), and a steady downward trend in 
score variation in four systems: New Zealand (14 points), Oman (34 points), Syria 
(19 points), and Tunisia (nine points). 
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Table 7.1 Standard deviations in student performance in TIMSS mathematics by education system 
at grade four 


Education system Year of TIMSS cycle 

2003 2007 2011 2015 
Algeria 89.56 
Argentina (Buenos Aires) 80.11 
Armenia 86.68 86.68 88.88 
Australia 80.86 83.32 86.30 83.47 
Austria 67.94 62.70 
Azerbaijan, Republic of 100.99 
Bahrain 89.89 85.00 
Belgium (Flemish) 58.95 59.60 60.80 
Botswana 89.34 
Bulgaria 82.65 
Canada 75.02 
Canada (Alberta) 66.06 64.81 
Canada (British Columbia) 71.31 
Canada (Ontario) 70.61 68.00 13.21 72.48 
Canada (Quebec) 65.48 67.35 60.21 66.30 
Chile 80.52 73.18 
Chinese Taipei 63.03 69.23 73.21 70.83 
Colombia 90.18 
Croatia 67.07 66.08 
Cyprus 85.39 80.71 
Czech Republic 71.46 70.38 69.86 
Denmark 70.83 710.76 75.15 
El Salvador 90.82 
England 87.41 86.04 89.38 83.73 
Finland 68.37 66.65 
France 74.34 
Georgia 88.43 89.84 86.81 
Germany 68.15 62.13 65.36 
Honduras, Republic of 83.61 
Hong Kong, SAR 63.39 67.13 66.41 65.64 
Hungary 77.25 91.16 89.79 87.97 
Indonesia 90.37 
Iran, Islamic Republic of 85.70 83.52 92.75 100.94 


(continued) 
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Table 7.1 (continued) 


Education system Year of TIMSS cycle 

2003 2007 2011 2015 
Ireland 77.95 73.13 
Italy 82.05 77.03 72.16 71.58 
Japan 73.75 76.08 72.30 68.73 
Kazakhstan 83.81 83.73 82.31 
Korea, Republic of 68.34 67.39 
Kuwait 99.30 101.21 101.06 
Latvia 72.52 71.90 
Lithuania 73.81 75.76 74.01 71.23 
Malta 77.69 
Moldova 87.33 
Mongolia 85.45 
Morocco 90.25 95.27 102.53 91.45 
Netherlands 54.62 61.35 52.96 56.01 
New Zealand 84.23 86.14 83.47 89.64 
Northern Ireland 85.89 85.72 
Norway 80.24 76.22 68.36 70.58 
Oman 104.07 100.66 
Philippines 109.71 
Poland 73.03 71.27 
Portugal 68.67 72.45 
Qatar 90.07 105.64 96.82 
Romania 105.35 
Russian Federation 78.25 83.37 73.75 72.72 
Saudi Arabia 100.04 91.67 
Scotland 77.54 78.93 
Serbia 88.80 86.82 
Singapore 84.22 84.15 78.17 86.01 
Slovak Republic 84.94 79.62 79.58 
Slovenia 771.95 71.40 68.51 68.74 
Spain 70.30 69.22 
Sweden 80.58 66.84 69.07 
Thailand 79.67 
Tunisia 99.59 110.81 94.79 
Turkey 100.53 95.24 
Ukraine 84.48 


(continued) 
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Table 7.1 (continued) 


Education system Year of TIMSS cycle 

2003 2007 2011 2015 
United Arab Emirates 98.56 105.34 
United Arab Emirates (Abu Dhabi) 96.81 108.25 
United Arab Emirates (Dubai) 89.60 100.64 94.15 
United States 76.27 75.33 75.58 81.49 
United States (Indiana) 65.45 
United States (Massachusetts) 69.77 
United States (Minnesota) TWA 
Yemen 113.59 110.14 109.92 


Notes Education systems did not necessarily participate in every cycle 


7.2.2 Inequality as Within-Country Variation I: The 
Influence of Teacher Factors on Student Variation 


We further examined whether teacher factors and student controls might account for 
the apparently random variation in overall within-country inequality in mathematics 
scores. Replicating the fixed-effects analysis employed in Chap. 5, we constructed a 
model with two student-level controls (books in the home, and language of the test 
spoken at home) and five teacher-level predictors (alignment, time spent on teaching 
mathematics, teacher education, self-efficacy, experience, and teacher gender). The 
purpose of the model was to explore whether within-country temporal changes in 
teacher human capital might account for score variations. Of particular interest 
was whether greater alignment with national standards and more time spent on 
mathematics might be associated with lower standard deviations in mathematics 
outcomes. Although teacher characteristics such as experience and education are 
conventionally treated as measures of teacher quality, content coverage and time 
spent on mathematics could also be viewed as metrics of high-quality instructional 
practices (although, of course, time and content are influenced by school policies). 

This analysis yielded fairly weak results (Tables 7.3 and 7.4). At grade four, none 
of the predictor variables were statistically significant, and, contrary to expectations, 
the direction of association between time on mathematics and alignment was positive 
rather than negative; in other words, inequality increased. The predictors also failed 
to reach the 0.05 level of statistical significance at grade eight, although changes 
in self-efficacy were significant at the looser 0.10 cutoff. However, self-reported 
preparation to teach mathematics topics had a weak and non-significant association 
with greater inequality. Unlike grade four, at grade eight curricular alignment and 
time spent on teaching mathematics were associated with smaller standard deviations, 
although with very weak t-values. 
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Table 7.2 Standard deviations in student performance in TIMSS mathematics by education system 
at grade eight 


Education system Year of TIMSS cycle 

2003 2007 2011 2015 
Algeria 83.52 
Argentina (Buenos Aires) 81.54 
Armenia 76.32 73.49 71.55 
Australia 84.08 65.91 58.05 83.23 
Bahrain 99.97 81.38 92.75 77.23 
Belgium (Flemish) 69.33 
Bosnia and Herzegovina 91.00 
Botswana 71.92 79.51 88.79 74.30 
Bulgaria 84.68 76.67 
Canada 79.87 
Canada (Alberta) 89.01 
Canada (British Columbia) 83.85 
Canada (Ontario) 73.09 66.75 78.29 88.38 
Canada (Quebec) 74.26 80.56 68.13 69.39 
Chile 78.32 70.86 91.84 
Chinese Taipei 87.34 90.23 76.62 78.32 
Colombia 74.82 
Cyprus 88.85 80.09 
Czech Republic 82.38 
Egypt 71.10 107.15 63.98 
El Salvador 71.18 
England 79.26 60.34 79.99 70.47 
Estonia 59.25 
Finland 84.74 
Georgia 79.43 83.60 77.80 
Ghana 76.58 101.60 72.44 
Honduras, Republic of 70.21 
Hong Kong, SAR 68.41 105.51 78.94 89.32 
Hungary 73.69 100.25, 72.82 83.58 
Indonesia 96.46 91.60 93.73 
Iran, Islamic Republic of 84.68 87.34 86.09 98.89 
Ireland 76.23 
Israel 85.42 102.21 92.07 78.64 


(continued) 
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Education system 


Year of TIMSS cycle 


2003 2007 2011 2015 
Italy 74.64 79.74 79.25 91.77 
Japan 81.49 80.33 65.66 94.94 
Jordan 102.44 93.36 99.75 83.08 
Kazakhstan 76.43 79.73 
Korea, Republic of 89.45 92.96 71.62 68.59 
Kuwait 70.04 82.40 
Latvia 91.62 
Lebanon 66.52 108.74 89.23 96.18 
Lithuania 76.74 79.23 67.76 90.68 
Macedonia 85.42 99.57 
Malaysia 77.66 63.09 70.58 61.84 
Malta 79.65 105.91 
Moldova 85.25 
Mongolia 64.85 
Morocco 105.91 85.55 76.98 84.47 
Netherlands 89.59 
New Zealand 83.93 94.70 98.21 
Norway 73.17 84.68 99.23 79.69 
Oman 74.75 78.95 108.64 
Palestinian National Authority 91.85 85.55 84.78 
Philippines 64.73 
Qatar 108.16 100.43 110.16 
Romania 101.86 81.13 93.47 
Russian Federation 84.10 70.48 86.12 67.52 
Saudi Arabia 96.88 85.88 75.44 113.08 
Scotland 89.84 88.39 
Serbia 86.90 93.35 
Singapore 77.05 88.76 82.36 80.32 
Slovak Republic 83.40 
Slovenia 69.75 70.84 64.22 79.96 
South Africa 97.18 98.56 79.84 
Spain (Basque Country) 91.95 78.41 
Sweden 93.39 94.08 73.95 102.02 
Syria, Arab Republic of 74.54 88.90 93.83 
Thailand 93.23 85.29 91.07 


(continued) 
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Table 7.2 (continued) 


Education system 


Year of TIMSS cycle 


2003 2007 2011 2015 
Tunisia 75.26 77.31 86.64 
Turkey 88.43 80.05 87.88 
Ukraine 70.05 65.52 
United Arab Emirates 96.13 102.21 
United Arab Emirates (Abu Dhabi) 81.71 86.11 
United Arab Emirates (Dubai) 82.13 69.25 87.07 
United States 71.96 89.18 105.41 97.90 
United States (Indiana) 98.54 
United States (Massachusetts) 91.04 
United States (Minnesota) 83.30 


Notes Education systems did not necessarily participate in every cycle 


Table 7.3  Country-level fixed effect estimates of the relationship of teacher quality to standard 
deviations in student performance, grade four 


Parameter Estimate SE p-value 
Alignment 15.57 8.93 0.09 
Mathtime 0.03 0.02 0.22 
Mathprep 0.88 3.57 0.81 
Books —5.56 7.61 0.47 
Lang —2.13 2.90 0.47 
Prepared —0.95 3.77 0.80 
Exp 0.02 0.39 0.97 
Tmale —25.00 14.23 0.09 


Notes Alignment = proportion of mathematics topics reported as covered by teachers compared 
with national expectations; Books = index (1—5) of number of books in the home; Lang = index 
(1—4) of testing language spoken in the home; Mathprep = index (1—5) of teacher education to 
teach mathematics; Mathtime = mean number of minutes spent on mathematics teaching per week; 
Prepared = index (1—4) of self-efficacy to teach mathematics; Exp = experience teaching in years; 
Tmale = index of teacher gender (female = 0, male = 1). A p-value = 0.05 or lower indicates 
Statistical significance 


7.2.3 Inequality as Differences Between High- and Low-SES 
Classrooms 


Instead of employing standard deviations as a measure of educational inequality, 
one alternative is to consider classroom effects. We calculated the variation in 
student mathematics outcomes for students who all had the same mathematics 
teacher (in other words, within-classroom inequality), and then ran a series of 
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Table 7.4 Country-level fixed effect estimates of the relationship of teacher quality to standard 
deviations in student performance, grade eight 


Parameter Estimate SE p-value 
Alignment —9.04 19.38 0.64 
Mathtime —0.01 0.09 0.91 
Mathprep 6.09 4.92 0.22 
Books —0.08 13.33 1.00 
Lang 8.16 19.70 0.68 
Prepared 13.29 7.53 0.09 
Exp 1.97 1.23 0.12 
Tmale 52.51 42.06 0.22 


Notes Alignment = proportion of mathematics topics reported as covered by teachers compared 
with national expectations; Books = index (1—5) of number of books in the home; Lang = index 
(1—4) of testing language spoken in the home; Mathprep = index (1—5) of teacher education to 
teach mathematics; Mathtime = mean number of minutes spent on mathematics teaching per week; 
Prepared = index (1—4) of self-efficacy to teach mathematics Exp = experience teaching in years; 
Tmale = index of teacher gender (female = 0, male = 1). A p-value = 0.05 or lower indicates 
statistical significance 


single-level within-country linear regressions using the standard set of predictors. 
Our main hypothesis was that teachers who spent more time on mathematics would 
be associated with smaller differences between students in their class, especially at 
grade four. 

These regressions also produced only very weak results (Tables 7.5, 7.6, 7.7, 7.8, 
7.9, 7.10, 7.11 and 7.12). There were few statistically significant associations, and 
none of these relationships were consistent across time; this finding raises serious 
doubts about the stability of these associations, even within countries. Further, where 
there was statistical significance, there was no consistent direction of association, 
which suggests that there is no general cross-national association between teacher 
quality and within-classroom inequality. Time spent on mathematics was only 
statistically significant in one system at grade four (namely Hungary in 2011), but 
(surprisingly) in eight systems at grade eight. In most cases where p < 0.05, the 
relationship between time and within-classroom variation in performance was in the 
expected direction; in other words, more time spent on teaching mathematics led to a 
decrease in inequality. The only positive and statistically significant relationship was 
for Moldova in 2003. The strongest result was that for Japan, where more time spent 
on teaching mathematics was significantly associated with lower standard deviations 
in student outcomes in both 2007 and 2011. 

Our second method of analyzing educational inequalities also relies on 
classroom-level characteristics, but instead of aggregating all classrooms together, 
we differentiated high- and low-SES classes. The key variable we used to define 
socioeconomic status was the common proxy variable, number of books in the 
home. Our approach for identifying a classroom as high- or low-SES builds on that 
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used by Schmidt et al. (2015), and Burroughs and Chudgar (2017), who both used 
interquartile differences. First we calculated the mean number of books in the home 
per classroom, and we then identified all of those schools above and below the 25th 
and 75th percentile, respectively. Finally, the average of classroom characteristics 
was taken for each key variable. Welch’s t-test was used to determine whether these 
differences are statistically significant, since it is not as sensitive to variation in 
sample size or variance between groups, unlike Student’s t-test (Derrick et al. 2016). 
However, it must be emphasized that our analysis may be vulnerable to Type I (“false 
positive”) error, since standard errors were calculated using the adjusted weight 
model (as discussed in Chap. 5) instead of by using jackknife standard errors. The 
jackknifing procedure was developed for use with the entire sample of schools, not 
a subsample as employed here. Another limitation is that often only relatively few 
classrooms are compared against one another. The results should therefore be treated 
with caution. 

As might be expected, our analysis showed large and statistically significant 
differences in mean student performance between high- and low-SES classrooms 
in nearly every instance. At grade four, the gap was statistically significant in all 
but five cases, and statistically significant and negative (richer classrooms posting 
lower mathematics scores) in only two cases (Armenia in 2007 and Saudi Arabia in 
2015). At grade eight, there was a statistically significant and positive advantage for 
high-SES classrooms in all but four cases. 

At grade four, as with other analyses, there were only a modest number 
of instances where the teacher quality differences between high- and low-SES 
classrooms were statistically significant (Table 7.13). The most powerful results 
at grade four were found for teacher self-reported preparedness to teach math, 
with statistically significant positive gaps (i.e., greater advantage for wealthier 
classrooms) in 15 instances, and statistically significant negative gaps in four cases. 
This inequality could be due in part to differences in teacher placement, but could 
also reflect biases in the instrument if teachers in advantaged schools were to 
have higher rates of professional satisfaction. However, there are some general (if 
non-significant) patterns. Pooling across cycles, there were 122 cases (educational 
systems across multiple years) where high-SES classrooms had more experienced 
teachers. Similarly, teachers in high-SES classes reported higher self-efficacy in 125 
cases. The other variables saw much more variability in the relationship between 
classroom SES and measures of teacher effectiveness. In approximately half the 
TIMSS countries, low-SES classrooms had teachers who reported stronger alignment 
to the curriculum, better education to teach mathematics, and spent more time on 
teaching mathematics than teachers in high-SES classrooms. 

At grade eight, statistically significant differences in teacher quality were more 
common, but also occurred in both directions. High-SES classrooms registered 
significantly higher teacher experience in 37 cases, but lower teacher experience in 
four cases. A similar result was found for teacher education (25 positively significant 
versus eight negatively significant cases), alignment (21 positively significant versus 
seven negatively significant cases), and self-efficacy (39 positively significant versus 
three negatively significant cases). The results were more balanced for time spent 
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Table 7.13 Number of education systems with statistically significant differences (positive or 
negative) in teacher quality metrics by classroom socioeconomic status, 2003—2015 


Variable Grade four Grade eight 

Positive Negative Positive Negative 
Exp 6 5 37 4 
Mathprep 6 6 25 8 
Mathtime 8 6 25 8 
Alignment 3 4 21 7 
Prepared 15 4 39 3 


Notes Exp = experience teaching in years; Mathprep = index (1—5) of teacher education to teach 
mathematics; Mathtime = mean number of minutes spent on mathematics teaching per week; 
Alignment = proportion of mathematics topics reported as covered by teachers compared with 
national expectations; Prepared = index (1—A4) of self-efficacy to teach mathematics 


on teaching mathematics (25 positively significant versus 28 negatively significant 
cases). The results were quite similar when all differences (not just those that were 
statistically significant) were considered. 

Although these results point to modest advantages for high-SES classrooms at 
grade eight and more equity at grade four, it should be remembered that the results 
were often quite inconsistent across years. In only a handful of cases was there 
a Statistically significant difference for the same country across multiple years. 
For example, high-SES classrooms had higher mean teacher experience in four 
cycles of TIMSS for Iran and three for Syria. For time spent on mathematics, in 
Chinese Taipei, more affluent classrooms spent more time on grade eight mathematics 
with statistically significant differences in three different cycles of TIMSS, while 
there were multiple significant and negative differences (namely where low-SES 
classrooms had the advantage) for four cycles of TIMSS in Singapore and three 
cycles in the United States. Teachers in high-SES classrooms also had reliably higher 
self-efficacy in Jordan in three cycles of TIMSS. 


7.3 Discussion 


An equity analysis of TIMSS data provides strong evidence that there is a broad, 
substantial, and enduring inequality in student outcomes. Cross-national analysis of 
within-country standard deviations demonstrates considerable variation in student 
performance, and students in high-SES classrooms generally outperform students 
in lower-SES classrooms. However, there is considerably less support for the 
hypotheses that there are important differences in teacher quality between types 
of classrooms, or that educational inequalities are based on such differences. Our 
analyses also raise important questions about whether teacher characteristics have 
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similar effects on students when cultural contexts differ. The variation in the size, 
strength, and direction of indicators between study cycles also raises genuine 
concerns about overreliance on a single year of TIMSS data when making inferences 
about effect of teachers on students. 

Having said that, our analysis of equity does highlight one important conclusion: 
policymakers and researchers should be careful about attributing the lessons drawn 
from one educational system to another. It is simply not the case that low-SES students 
have less experienced or educated teachers in every national context (although in 
many they do), as many studies have found in the United States. In some educational 
systems at some grades, students in lower-SES classrooms may have the teachers that 
are more experienced and better prepared to teach. But other lessons do have more 
general applicability. For years now, a growing body of literature in the United States 
has suggested a straightforward equation of easily observable teacher characteristics 
are a poor indicator of quality instruction, absent of more robust statistical models 
and controls. The TIMSS data suggests that this lesson is broadly applicable to many 
countries. Equity remains an issue of vital concern, but an exclusive reliance on 
policies like improving teacher alignment or time spent on teaching mathematics 
may be unlikely to reduce these inequalities and improve student outcomes. 
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Chapter 8 M®) 
What Does the TIMSS Tell Us About Bcchiog 
Teacher Effectiveness? 


Abstract The central aim of this report was to investigate what international 
comparative assessments could reveal about the role of teachers in influencing 
student outcomes. A series of analyses attempted to identify potential trends in 
teacher quality and instructional metrics over time and their relationship with 
student achievement, but found no strong evidence for consistent predictable 
relationships between commonly-employed indicators of teacher effectiveness and 
student outcomes (both within and across educational systems). Further, although 
inequality in student outcomes remains pervasive across education systems, teacher 
quality is less subject to inequality, and, indeed, many countries have seen a 
rise in formal preparation to teach mathematics over the twenty years of IEA’s 
Trends in International Mathematics and Science Study (TIMSS). While improving 
conventional measures of teacher quality may not have a significant impact on 
educational inequality, there is some evidence that increasing the average amount of 
time spent on teaching mathematics may reduce inequalities in student achievement. 
Researchers and policymakers should be extremely cautious about applying the 
associations identified in one education systems to a very different educational 
context. The results indicate that teachers with similar experience, credentials, and 
instructional strategies may regardless produce quite different results, indicating that 
parents, policymakers, community institutions, and cultural context likely play a 
powerful role in determining student outcomes. 


Keywords Educational equity - Student achievement - Teacher quality - Trends in 
International Mathematics and Science Study (TIMSS) 


The research questions posed in the first chapter of this work were as follows: 


(1) Are there identifiable trends in teacher quality and instructional metrics over 
time? 

(2) What are the relationships between student achievement and different types of 
teacher quality and instructional metrics? 

(3) How stable are these relationships across time and by statistical method? 

(4) What are the relationships between student equity and different types of teacher 
quality and instructional metrics? 
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Chapter 4 focused on the first research question. Country-level descriptive data 
at grade four revealed that some teacher characteristics (e.g., teacher education) 
were fairly stable across time, but there was wide temporal variation in teacher 
behaviors like instructional alignment with national curricula and time spent on 
teaching mathematics. However, there were important differences in these variables 
between grades four and eight; at grade eight there was far greater variation in 
levels of teacher education, but less variation in time spent on teaching mathematics. 
Meanwhile, many countries saw increases in teacher characteristics like formal 
preparation to teach mathematics, perhaps in response to policy initiatives prompted 
by the TIMSS results. At both grades, there were many countries that demonstrated 
an increase in teacher self-efficacy over the twenty years of TIMSS. At both grades, 
some countries exhibited consistency in levels of teacher experience, while others 
demonstrated wide variation in this variable between cycles of TIMSS. 

Research question 2 was addressed in Chaps. 5 and 6. Chapter 5 identified 
relationships between teacher quality measures and mean student outcomes 
using several statistical approaches (pooled within country, students clustered 
within classrooms, classroom-level means, and country-level fixed effects models). 
Chapter 6 used multilevel structural equation modeling to explore the interactive 
effect of teacher characteristics and behaviors. The key takeaways from all of these 
analyses were: 


(1) There were no generally valid relationships between teacher characteristics and 
student mean outcomes, rather there were dramatically different relationships 
from one educational system to the next; 

(2) Of the two teacher behaviors associated with opportunity to learn considered in 
this study, time spent on teaching mathematics was the only behavior identified 
as statistically significant across countries in the fixed-effects model; and 

(3) Teacher instructional alignment with national curricular expectations has 
exceptionally weak associations with student outcomes. 


Chapter 5 also considered research question 3, examining the within-country 
consistency of statistical estimates across time and by analytical method. These 
analyses indicated that there were often differences in the strength and direction of 
many associations between teacher variables and student mathematics performance. 
This instability should give pause to researchers and policymakers who may be too 
quick to draw conclusions from one cycle of data or derived using one statistical 
method. The sensitivity of standard error estimates, which are critical to determining 
whether a relationship is statistically significant, should likewise warn researchers 
against neglecting the complex sampling design of TIMSS. 

The topic of student equity, addressed in research question 4, was the subject of 
Chap. 7. Consistent with other research (e.g., Schmidt et al. 2015), our analysis 
demonstrated that there was considerable educational inequality, whether equity 
is understood simply as variation or as differences between high- and low-SES 
classrooms, and that this inequality varies considerably between education systems. 
However, there is was only a very weak relationship between this inequality and 
conventional measures of teacher effectiveness, whether those metrics were related to 
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teacher preparation or to teacher behaviors. In short, at least according to our analyses, 
improving conventional measures of teacher quality may not have a significant 
impact on educational inequality. However, there is some evidence that increasing 
the average amount of time spent on teaching mathematics may reduce inequalities 
in student achievement. 

The research we have presented here is certainly extensive, but by no means 
conclusive. TIMSS operationalizes teacher-level variables in very specific ways, 
distinct from those in other research that have suggested stronger associations for 
teacher effectiveness measures. Teacher-reported preparation to teach mathematics 
topics is a less precise measure of teacher content knowledge than tests of that 
knowledge, which some studies have found to be related to student outcomes. 
Similarly, instructional alignment is based on very different assumptions from 
curricular intensity. Beyond this, there are other methods for exploring the potential 
of teacher instruction; for example, alignment and teacher preparedness is defined 
with a summary index, rather than being matched to more specific dimensions of 
student learning. 

It should be acknowledged that the design of TIMSS carries with it certain 
limitations. Because TIMSS only selects one or two classrooms within a given 
school, it makes it extremely difficult to differentiate teacher from school effects, 
or to identify within-school, between-classroom heterogeneity. The cross-sectional 
nature of TIMSS, and the lack of consistent country participation across cycles, also 
presents challenges. 

If there is one lesson that should be absorbed by readers of this report, it would 
be that researchers and policymakers should be extremely cautious about applying 
the associations identified in one education systems to a very different educational 
context. Simple transference of policy ideas that have enjoyed apparent success in 
one educational system can yield unexpected (or even disastrous) consequences in 
another. International comparative research is thus an extremely fruitful way to test 
the universality of given approaches. Further, this study suggests that the search for 
broadly applicable, reliable, easily collected measures of teacher effectiveness is 
likely to be long and difficult. 

On a more positive note, we found that replicating statistical models across 
different cultural contexts and time periods can be extremely fruitful. There is a 
strong temptation for researchers situated within a given educational system, or 
using a given set of data, to draw overly broad generalizations about the universality 
of their findings. Large-scale analyses employing studies from different countries, 
and replicated across time, can serve as a useful check on the robustness of scholarly 
work. 

Finally, our findings pose a challenge to those who would place too much 
responsibility for perceived educational ills on teachers. At least in the United States, 
there has been a tendency among some policy activists to present “bad teachers” as 
the reason for poor educational outcomes. The results of this study suggest that 
teachers with similar experience, credentials, and instructional strategies produce 
quite different results, which could mean that adequate cross-national measures of 
high-quality teaching are lacking, or that teachers’ effectiveness is conditioned on 
other circumstances. The totality of the educational system itself, and the social 
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structures it rests upon, powerfully shape student outcomes. Accordingly it is a 
profound mistake to place too high a burden on teachers (or schools). Teachers 
are essential to the educational project, but parents, policymakers, community 
institutions, and cultural context may also play a powerful role in student outcomes. 
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Appendix A 
Detailed Country-Level Means for Key 
Variables 


We here present mean teacher and student level results for the key variables 
identified in Chap. 3 for each educational system (here referred to as countries) and 
for all TIMSS cycles between 1995 and 2015 (Tables A.1, A.2, A.3, A.4, A.5, A.6, 
A.7, A.8, A.9, A.10 and A.11). The information is summarized and discussed 
further in Chap. 4. 


© International Association for the Evaluation of Educational Achievement (IEA) 2019 141 
N. Burroughs et al., Teaching for Excellence and Equity, IEA Research for Education 6, 
https://doi.org/10.1007/978-3-030-1615 1-4 


& 
s 
5 
> (penunuos) 
g (10'p) €6r (Z0'0) 80°0 (66°0) tr (PL'S) 11 6P7 (S00) Pre puepoog 
& (L6'€) Tbr (10°0) 700 (LS°0) €6°17 (19°L) OLIVE (10'0) 60°T (S0'0) Or'Z [esnyl0g 
a (86°C) OLY (€0°0) 810 (8L'0) ZEST (Op) 80°81 (100) PIT (ZOO) LLE AG AMON 
s (€p'b) 69 (€0°0) LZ0 (09°0) 80°rT (SZ'b) 69° ETT (100) LIT (v0'0) ILE pueleoZ MON 
2 (p0'€) 6rS (70'0) SS°0 (860) SILT (79°) 61087 (ZO'O) 811 (S0'0) Sve SpuepoyyoN 
a (€S"p) 66 (10°0) €0°0 (98°0) Z9'81 (69°7) €8°607 (100) ITT (€0'0) 68°€ BIATeT 
E (L6'€) ISE (€0°0) 9F'0 (0S"0) $9°6 (€8°T) OF'ELT (€0'0) £9°T (S0'0) 99°7 weany 
3 (18'T) 18 (€0°0) PEO (Z9°0) p8°LT (V8'T) TVZLT (100) ITT (ZO'0) 9EE Jo oyqnday ‘ea1oy 
S (06'T) L9S (€0°0) PEO (ZS‘0) P0'9T (81°1) 06°€7Z uedes 
- (Erb) SOS (Z0'0) 200 (€fD IS 91 (pr L) 6° LET (100) 61'T (S0'0) O€E [oes] 
a (€S°€) ETS (€0'0) 870 (S8°0) 99°81 (8p) €6°0SZ (100) ETT (0'0) 97E pueyely 
< (98'p) L8E (v0'0) €r'0 (€S°0) €6°0I (v0'0) SOT (0'0) SST Jo oygndoy orureysy ‘uely 
s (L7'€) ESP (ZO'0) €1°0 (ZO'T) IZ FI (6b'€) 06°061 (100) PIT (Z0'0) 69°€ pueyoo] 
a. (IS°€) 17S (10°0) L0°0 (p90) 81 (6€°@) SLTIZ (10'0) 60°T (v0'0) Z9°€ Aresuny 
< (10"p) LSs¢ (€0°0) O€°0 (S8°0) EL'vI (LL'b) IL°67Z (€0'0) 09°7 UVS ‘suoy suoy 
(Tvs) €94 (€0°0) Zr'0 (66°0) L8°ST (OT'€) 6S°S6T (100) €1'T (0'0) €8°7 9090.1) 
(SEE) H8r (ZO'0) 70 (89°0) 00°ST (9I'p) TI ELT (100) TT (€0'0) LEE puelsuq 
(00°) Irs (Z0'0) 80°0 (v8°0) SEIT (CL'D) pl St (10'0) 90°T (€0'0) LOE aqndey yoeza 
(€Z'€) SLY (v0'0) TE0 (86°0) L891 (81°C) 97 PHT (100) 9T'T (€0'0) POE studéD 
(8€"€) 90S (Z0'0) STO (vS'0) E781 (Sb) 60° P97 (100) LIT (€0'0) Z9E epeue) 
(L8°) TES (€0°0) 770 (L'0) 86°81 (€8°0) I1°ZO7 (ZOO) €7T (70'0) TE eiasny 
(6r'€) S6r (v0'0) O€°0 (8r'0) LOST (9b) LOEST (100) ZIT (Z0'0) 86°€ eye.nsny 
QOURULIOJ.INg oyeury, dxq ou IRL suey] syoog Anunog 


142 


p Opels C66I SSINLL ‘Sorqeuea Aoy JOJ suvoul [oAg[-ANuNOD T'V IqUL 


143 


Appendix A: Detailed Country-Level Means for Key Variables 


SoneUsyeU Ul B1OIS SSILL JUopms uRoW = soUPUIOJIOg ‘(| = ofew ‘Q 
= 9[PUIDJ) JOpUSS Joysed} JO XOpUl = oeULL ‘sIV9X UI SuryoRs) souatIedxa = dxq ‘yoom Jod Suryoes} sonewuoyyeul UO JUads soynuTU Jo JaquInU UBOUT = SWIRL 
‘ouloy oy) ul Uayods asensur] Suns} Jo (p—][) Xopul = Suey] ‘owoy oy) Ul syoog Jo Joquinu jo (¢—[) xopul = syoog ‘sesoyjuared ur sIOLIa prepuerjs sajoy 


(v6'7) 81S (Z0'0) O10 (€L'0) FIST (I'S) rr'6rZ (Z0'0) 0@'T (70'0) 19°€ Soye1g Poul) 
(88'p) LOV (S00) 170 (OL'0) Lv'Il (pZ'p) OS PLZ (€0'0) SST (700) 98°T pueyreyy 
(81'€) Z9r (Z0'0) 400 (LL'0) TL'81 (P0'2) 08 TEZ (100) ZV (€0°0) 6V'€ BIMSAOTS 
(ESP) 06S (Z0'0) 810 (LL'0) VL'LT (99°0) €6°LZE (100) 88°T (70'0) LIE arodesurg 
QOURULIOJ.10g oyeut dxq ou IRL suey syoog Anunogd 


(penuyuos) TW age. 


Appendix A: Detailed Country-Level Means for Key Variables 


144 


(ponunuos) 
(€S°7) 6LY (Z0'0) €0°0 (78°0) SS'8T (€0'0) LEZ (VOT) OET (80°0) OI'P (ZO'0) 6€T (€0°0) 00°E (10°0) TS°0 BIUSACTS 
(LS°S) P6S (€0°0) 810 (88°0) €€ 01 (€0'0) TL'7Z (68°T) 8Z€ (P10) SSE (€0°0) LET (¥0'0) 76°7 (10°0) 19°0 arodesurg 
(OI'€) 06r (Z0'0) LOO (88°0) Lv'ST (v0'0) 9L'7 (98°9) 967 (¥0°0) ZO" (Z0'0) LET (v0'0) 8T€ purpioog 
(QL'b) TES (10'0) 10°0 (pL'0) 09°0% (S61) LOZ (S00) ES (S00) Tel (70'0) €6°7 uonelopay uvIssnYy 
(S0'8) 8S€ (€0'0) €1°0 (68'0) 8€°€1 (700) 8E°7 (00°8) 1S@ (600) FOr (€0'0) 80°€ (S00) Z6'T sourddiyd 
(612) Ist (Z0'0) 61:0 (€6°0) OL'ST (€0'0) 19°7 (v8°7) PLT (00°0) 00°T (10°0) O€T (€0°0) PTE (10°0) €5°0 AGMION 
(S12) €6r (Z0'0) 610 (€9°0) 87° TT (ZOO) 89°7 (00'P) 817 (v0'0) LIP (Z0'0) LET (€0°0) O7E (10°0) 85°0 purleoZ MON, 
(LV) Obs (S00) 9€°0 (PID) £791 (700) 9€°7 (06'S) OLZ (870) Sor (Z0'0) €€I (70'0) 60°€ (10'0) 09°0 Spur] oujoN 
(p0'S) LrE (70'0) 790 (pL'0) 60°ST (110) 077 (90°) €SZ (L0°0) ¢7'T (L0'0) Lv'z (€0°0) 19°T 099010] 
(SL'p) vOS (10'0) 200 (S80) 611% (€0'0) LEZ (8S°€) SIZ (S10) ELE (€0'0) 9€ (70'0) 677 PAOPIOW 
(IL) vEs (10'0) 10°0 (99'0) $881 (Z0'0) LZ (OE) E17 (70'0) Sor (Z0'0) 071 (70'0) L9°Z (10'0) 9S°0 vruenyy Ty 
(€8°7) 9ES (100) 10°0 (98°0) 71:07 (€0'0) Sv’Z (SPT) 007 (€1'0) 60° (€0°0) TET (€0°0) STE BIA] 
(LST) 9S (p00) LEO (80) ZL'81 (€0'0) LOZ (L6°T) POT (L0°0) 96°€ (10°0) TTT (Z0°0) 9L°7 (10°0) 19°0 uedes 
(g9°€) €0S (10'0) 700 (85°0) ZOIZ (€0°0) LEZ (27'S) SZE (00°0) 00°T (10'0) E71 (€0'0) 79°7 (10'0) LS°0 Ayeay 
(OI'b) 68 (S0'0) 6r'0 (89'0) 67°91 (€0°0) LEZ (v6'r) £07 (910) 98°7 (60°0) 60°7 (S00) 78°T (00°0) €S°0 Jo oyqndoy orweysy “ues 
(LV'€) 67S | (ZOO) 90°10 | (810) EST) ~— (HOD) @H'Z (61°2) 661 (100) ITT} = (00) STE | (100) 6S°0 Aresuny 
(ZI€) SLS (700) L70 (60°T) €v'€T (v0'0) Lv’Z (66'S) 9€7 (110) €6°€ (€0°0) 8LT (v0'0) €r°7 (10°0) 09°0 UAVS ‘Buoy suoy 
(IL'€) TEs (p00) L70 (96°0) €v'7I (67'b) EOE (L0°0) 6LE (ZO'0) ¥T (S00) PE (10°0) 85°0 pueysuq 
(Sr'Z) OI (€0'0) 170 (ZL'0) OL'O1 (Z0'0) 18°7 (S61) Ibz (€0'0) 81° (Z0'0) Srl (€0'0) €8°7 (10'0) 9¢°0 smud&D 
(QL'1) 9s (€0'0) 070 (690) 16°01 (€0°0) 677 (Lyd) 8r1 (O10) ZS"€ (Z0'0) Lol (70'0) 88°7 (10'0) $s°0 Jodie, esouryD 
(Iv) 90S (Z0'0) LOO (160) 6rLT (€O'0) ELT (LV'6) @ZE (L0°0) 98°€ (Z0'0) vr'T (€0°0) SL'Z (10°0) 750 (Seqond) epeurD 
(98°€) TIS (700) 770 (60) TT€T (€0'0) ELT (vS°S) 9b7 (60°0) 89°€ (€0°0) EST (S00) TEE (10°0) LS°0 (orejuQ) epeueD 
(6L'T) TSS (€0°0) 770 (L9°0) €1'9T (€0'0) 7S7Z (IT€) €T€ (v0'0) 9E" (€0°0) TST (€0°0) 00°E (ystwo]J) wNIsjog 
(L8°€) 66P (v0'0) S70 ($80) 79°9T (€0'0) 9L'Z (gs°S) 097 (v0'0) TIP (€0°0) TET (S0°0) Pre eyensny 
(8r'€) 9S (€0'0) Z10 (060) LO0z (€0°0) 69°Z (LS°€) 8E7 (90'0) LO’E (Z0'0) 71 (S00) €6°7 eIUSWUTYy 
QOURULIOJINg oewlL, dxq poredaig OUI daidyeyy suey] syoog qUOWUsITV Anunodg 


p Oprels COOT SSINLL ‘Sorqeuea Aoy JOF suvoul [oAg[-ANuNOD ZW IqUL 


145 


Appendix A: Detailed Country-Level Means for Key Variables 


SONeWIOYIeU UL B09S SSIALL JWopnys uRoW! = soURUIOJJOg “(] = o[PUI ‘CQ = aRWOJ) Jopuss Joyore} 
jo Xopul = oyeuLy ‘seoA ul SuryoRa) sousLIodxo = dxq ‘sonewmoyjeU yors) 0) Aovoyjo-j[as Jo (p—]) xopul = poiedaig ‘yoom Jod Suryors) sonewoyjew uo juods soynuru Jo Joquinu 
UROL = SWIYIRYAY ‘soNPUIOYJeU Yoo} 0} UOBONpe JoYoea} JO (C—T) xopul = dasdyyeyy ‘owoy ou) ur usyods oSensue] SuNsa} Jo (f~—-]) XOpul = Sur’] ‘owoY sy) UT Syoog Jo Joquinu 
JO (S—-[) xopul = syoog ‘suoneyodxo [euoNneU YIM posedur0d siayoeVa} Ag Posdaod se poyodas so1do) sonewoyjew Jo uoNJodod = jUOWIUsITY sazoyy ‘sosoyuored UI S100 prepueisS 


(SEL) 8LZ (€0°0) PL'0 (670) 69°6 (€0'0) O€ 7 (09°b) SET (Z10) 86°7 (90°0) 8L'T (90°0) Z6°T (10°0) 7S°0 usa A 
(€6'7) EES (v0'0) Z1°0 (10'0) 78°7 (Z0'6) SET (€0°0) 96°€ (Z0'0) v7T (0'0) ITE (euRIpUy) sayeig poyuA 
(OZ) 81S (Z0'0) S10 (pS‘0) OL'ET (Z0'0) LL’Z (LYS) Sp (+0'0) 96°€ (Z0'0) trl (€0°0) 00'€ soyeyg pour) 
(69°b) 6€E (p00) 7S'°0 (9L'°0) v0'8I (v0'0) E77 (€8°7) TET (v1'0) LET (90°0) 81°7 (S0°0) SO0°Z eIstuny, 
QoURUIOJ.INg grew dxq poredaig OU daidyey sur] syoog qUOWUsITy Anunog 


(penunuos) ZW aqeL 


(ponunuos) 


S (ZED LES} (00°0) 00'0 |: (89'0) B~z | (COO GE] (OS) 80z| (CO'O) ~r| (COO) IVT] (E00) So’E| _(10'0) 09°0 BIA] 
E (99°€) 91¢ | (0.0) Z1'0| (Or) 6£'8 (S00) ove} — (E19) 99 (600) 69°€ | (HOO) SLT} (P00) S97} (100) $9°0 WeMny 
> (L6°9) 6PS | (200) 90'0|_—- (€6'0) 6F'8T (€8'7) 6€Z| (OO) Ssr} (E00) EZT} (LOO) ZZ ueIsyeZey 
M (80°72) 895] (OO) SEO} (LO) L381} (S00) S97) (81) €0Z) (00) 96} (100) GOT] (ZOO) ILZ| (000) 85°0 uvder 
& (60'€) LOS} (100) €0'0} + (€9'0) 89:07 | (SOO) PZE} (STH) 6zE| OOO) ODOT} (MOO) ITT] (E00) 9L7| (100) SSO Aeay 
g jo oqndoy 
S (WO'r) ZOv| (coo) LYO} (eso) SzoI| (Soo) TOE} (ZLy) 061) (STO) OEE} (S00) 661] (00) P8T| (100) £50 ones] “Ue A] 
3 (is'6) 01S} (ZOO) S00} (90) IL7z| (coo 67E} (SO7 I8T| GOO) Lor} (MOO) ZIT] (OO) CoE] (100) rr0 Aresuny 
3 (Zs'¢) LO9| (POO) TO} (20) 9071} (OO) 967) (Hes) 6E7) OD) Zr} (ZO) 09T| (00) 787} (00) cL'0 UVS ‘Suoy Suoy 
2 (OV sz} (OO) STO) CLO) SSIzZ| (COO) Ee7) CLD E67} (coo) cor) MOO) EET} (ENO) EOE) (10'0) 090 Aueuiop 
5 (6z'p) 8ev| (00) 100} (60) EEsz| OOO) ISE} (CEzZ9IZT| COTO) Str} (ZO) L7T] (S00) 887) (10:0) 19:0 BIBIOIH 
S (960 Ips} (coo Teo} (SLo)080T| (coo) LLE} (LZ) 68z| (OO) 98E} (WO) TET] OO) Eze] (M00) SSO pueysug 
a (LT'p) O€€| (HOO OEO}] CIO) L7rI| (SOO) ITE} (LE) Tz] (C10) 987} (ZO) ZT} (E00) 08T| (100) 19°0 Jopeares Ta 
2 (9rd) €7S| (POO) TSO) CTT) 6E9T} (E00) 78 (991) L81| (OO) SvTE}] (ZOO) TET] (OO) TOE] (100) OS'0 yseuusd 
« (PLD 98h} (200) 600} (660) 8TZT| (SoO)OEE}] (8L0) Ez] GOO) OSE} (00) 07T| (00) 667] (0'0) 85°0 ayqndey ysez5 
x (40'S) Sse} (OO) PTO} (Er Drrel| (ooorEe} ro) TZz7| (IO erEe}| (ZO) EET] (E00) 107) (100) ISO BIqWO]OD 
3 OLD 9LS| (oo) S70} COLO) SLIT) CLO) 8rE} Ore) LT) (CIO trE}! (ZO PLT] (E00) Sxz| (M00) z75'°0 rodre, esouryo 
= (oo) 6Is| (COO ETO} CMO) rLSsT| (OO) IKE} (HBS) LZ] (00) 68E} (ZO) 8rI| rod) 167] (M100) ss0 (sagend) epeued 


(962) TIS | (E00) 910} (O80) ST'EL| (E00) ILE (sp'9) SL7| (LOO) I8'E| (E00) LET] (S00) L7E} (100) 7S°0 (oLeUQ) epeurD 
(eiquinjod Ysnig) 


(pL SOS} (00) STO} (SLO) LEST| (E00) 7E} (6ZH) OPT] (0'0) 88E} (EOD) OST] (E00) 6ZE] (100) rr'O epeurD 
(06°27) SoS | (€0°0) 0710} (LO) PO'FT | (E00) Z9'E (16€) €hZ| (LO) PLE) (EO) SFT} (OPO) 67E} (TOO) 9S°0 (eyeqry) epeurD 
(S61) Sos} (ZOO) ITO) (99:0) 7617} (P00) ITE (881) 60 | (€0°0) 96°E | (ZOO) IhT] (E00) 987} ~—(TO'0) ESO visny 
(sre) 91S} (€0'0) 070] (S60) LOT | (P00) LSE (069) 997 | (€0'0) €0'F | (ZOO) TET] (COO) IVE} (100) 090 vyeysny 
(9% b) 00S | (E00) STO] (780) LEZ} (90'0) 06% (v7S) 87Z| (SOO) EE] (ZOO) L7T}| (SOO) LLZ} (T0'0) 85°0 vIUouLy 
(47S) 8LE| (OO) ZrO] (SO)OSLT) (SOO) EEE] (SEH) 967} (800) IST} (900) LZTZ} (O00) OLT} (100) 9r0 BLIOS[V 
OURULIOJIOg g[eulL dxq paedaig OWE daidyyeyy suey] syoog quowusiTy Anunog 


146 


p pris £007 SSINLL ‘Sorqeuea Aoy JOF suvoul [oAg[-ANuNOD ¢°Y IQUL 


SONVUIYJLUT UT aI0NS SSIILL Jepnys uvour = souvuropiog ‘([ = afew ‘—Q = a[ewoy) Japuads Joyo} Jo xopul = o[eULy, ‘sIeoA ur 
Suryors) souotiadxe = dxg ‘soneuroyjeur yore 0) Aovoyjo-Jfas JO (p—]) xopul = poredarg ‘yaom Jad Suryoray sonewoyeut uo yuods saynurur Jo Jaquinu UvoUT = oUINYRY 
‘soNBUTOYeUL Yor} 0} UOTVONpa JoyoRa} Jo (G—[) xeput = dardyyeyy ‘owoy oy) ut uayods a8enZur] Burysaq Jo (p—]) xepul = Suey] ‘ouroy oy} Ur syooq Jo Jaquinu Jo (S—T) 
XOpUI = syoog ‘suonejoedxo JeuoNeU YIM poredurod ssayora} Aq posoAoo se poyodar soidoy sonewmayjeu Jo uonodoid = JuowusiITy ‘sosoyjuared ur s1oLe prepurys salon 


147 


(LO'9) p27} (OO) FLO) (LS0) PSZI| (HOO) STE (078) 0% | (IO) S7Z| =O) HT) GOO) ILT| (100) 790 woule x 
(eJOsouUTy) 

(18's) pss} (900) STO) (ETI POLIT) (POO) LLE| (oEDBSZ| (800) E07] (00) SET] (OO) IZE} (ZOO) 8S°0 sareig pau 
(snosnyorsseyy) 

(Ls'€) TLS| (£00) LO'O| +=(S60) SO'EL| = (ZOO) BBE] CHIEDErPE| (800) See} (COO) TET} (900) SE} (ZOO) 90 saweig pou, 

(IS) 67S | (ZOO) 710] (Sr0) 88'ET | (ZO) BLE (66'S) 687 | (P00) 88'E | (ZOO) TT] (E00) 667} (TOO) 890 sareig pau 

% (reqnq) soyerrrg 
2 (60°0) rrr} (S00) 070) (S60) PZT} (90'0) SSE (css) 8€7|  (ZE0) 97E| (SOO) BT'Z| (COO) TL'Z| (100) TS'0 gery pou 
5 (682) 697| (000) 00'0| (190) L77| (H0'0) 9E'E (ce) 8L1| (00) 80°7| (S00) NLT] (E00) 9L7| (100) rS°0 oureryy) 
a (Cp) LzE| (COO) PEO] (SSO) LTLT| (OO ITE} (LOZ coz} (ITO) SLT] (E00) 687%} (00) 96'T| (100) $90 erstun, 
Z (Lg) €0S| (€0°0) 810) (280) 66LT | (€0'0) 8r'E (09'€) SLT| (900) OFF) (COO) 8ET] COO) IZE} (100) 75°0 uspamg 
& (Is'D cos} (100) Z0'0| (290) bY'OT | += (H0'0) OTE (rD 92% | (00) rrr] (200) THT] (E00) 98'7| (100) S90 BIUSAOTS 
g (Sr) 96h} (200) 90:0} (OL'0) POST | ~—(H0'0) 067 (€r'0) S@Z| (00) 06€| COO) SPT] (E00) EL7| (100) 6S°0 ayqndoy yeao[s 
S (OL'€) 66 | (E00) E€'0} (09'0) 78°6 (€0'0) 19°€ (eDsze| (200) 977] (200) O€ Z| (ZOO) TO'E| + (100) 190 asodesulg 
3 (C77) vor | (700) 80°0 | (060) SL'FT | (€0'0) LL'E (IV'y) L8Z| (S00) Z6'E| (ZOO) EET] (HOO) GO'E} (TOO) 85°0 purpoog 
asi (Cob) prs | (100) 10'0|_— (vS'0) OS TZ (LT) r6l| (S00) 9F'7| (00) CET] (S00) 167 uoresope, URISsNy 
E (€O'D) 96Z| (000) 80°0|_ (€0'0) ¥8°8 (00'0) 6r'€ (670) SIz| (00) ZI'E| (MOO) E8T| (ZOO) TOE Teed 
B (6S) €Lh| (200) 810) (80) 69°91 | (€0'0) 69'E (8L'€) 781. | (000) OO'T| (100) 6z7T| (EO) LO'E| (100) 650 AB MION 
. (8€0) tor | (200) STO} (SO) SHIT] (€0'0) ore (9L'0) 9% | (E00) 00°F | (ZOO) OF T| (ZOO) LT'E| (100) 9S°0 purjeoZ Mon 
a (rd ses} (00) ZrO!) (OT) 6T8I| (S00) 9E'€ (69's) s9z| (LT'0) 9L'v | (€0'.0) OFT | (HO'!D) SZ]: (TO'0) LEO spurpoyjeN 
B (Ly) Ive} (700) OS'0| + (19°0) OE 9T | (S0'0) Er'E (19°€) SLZ| (800) Lv'T| (400) 97} (900) LT} (10°) €9°0 Od90I0JA 
< (so'r) 9€h| (E00) ZO} A8O)OSLT| OOO) ITE] (rr) €or} C10) 907] (OO) tPT} (€0'0) S6T| (10'0) 0¢°0 erOsuoT 
5 (6€2) OES} (100) Z0'0| + (9S°0) p8'0Z| (S0'0) 767 (LL) 807| (S00) Loe} (100) OZ T | += (0.0) SST} (100) SSO eroenyyry 
5 QOURULLIOJ.I9g grew L dxq poredaig OUI yeYL daidyyey sur] syoog quowUsITy AQunod 
< (penunuos) ¢°y aq 


148 


g (ponunuos) 
< (09°) LZS | (€0'O) 670} (€9'0) EETZT | (€O'O) SSE} (TH) LyZ| (SOO) 66'E| (TO'O) 8I'T | (HO'0) SO'E | (100) 6S°0 pueyoly 
a (092) Iv} (€0'O) 8E0} (E90) HLT | (C00) POT | (Z'P) OST} (SO'O) OSE} (100) 99°T |) (HO'O) ITZ} (10'O) Ev'0| Jo oNQndoy orweysy ‘ue 
Ny (6€'€) STS} (100) P00} (L'0) 96°EZ| (HOO) POE | (97'S) EPZ| (EDO) LOE} (OO'O) EOI | (HO'0) TOE | (100) Sr’ Aresuny 
& (EE) ZO9| (POO) ZV'0| (LL'0) Gr'PT | (90'0) POE | (HSh) OST) (80'O) 8Z'F}| (TO'O) BET | (HOO) Z8'Z | (100) 6E'0 UVS ‘Suoy suoy 
a (€S°S) 96€ | (POO) SEO} (¥6'0) TE FT| (100) PZE | CIT TD vee] CTO) LEZ} (100) OTT | (€0'0) 69'T | (100) Ss"0 Jo ognday ‘semnpuoy 
S (€7'7Z) 87S | (€O'O) 610} (S80) 67°6T | (€0'O) OTE} (EOD) SPZ| (8O'O) ET'V} (TOO) IZ | (O'0) LIE | (100) zS°0 Aueuliay 
2 (69°€) OSP | (ZOO) 0'0| (Z8°0) 96°72 | (SOO) 60"E | (TS°9) 79Z| (LOO) EZ'F| (ZOO) OZT | (H0'0) 0O'E | (10°0) T9°0 BIBIOID) 
a (LET) SPS | (POO) LZO} (S90) TILT | (€0'0) OSE} (80H) 8IZ| (OO) 86°E| (TOO) ITT | (€0'0) 6Z'E | (100) ZrO pueyury 
E (Lv) TPS | (OO) S70} (€8'0) TOZT | (€O'O) BLE | (OEY) 86Z| (80D) Z8°E | (ZOO) IZ | (HO'0) 90'E | (100) LS"0 purpsuq 
3 (LET) LES | (POO) ZH} (EL'O) ELST | (ZOO) SBE} (LOZ) 98T | (600) OPE} (TOO) OZ T | (€0'0) 967 | (10°0) 0S'0 reused 
S (672) ITS | (ZOO) S<0'0} (80) 9L'8T | (SOO) TEE] (EPH) BPZ| (LOO) S9°E| (TOO) PIT | (€0'0) LIE | (100) 79'0 aqndey yoez5 
- (261) 06h | (100) €0°0|}  (L'0) SL°07 | (E00) €9°7 | (BH'E) S7TT| (EOD) HI'y| (TO'O) ST'T| (€0'O) Ssz| (100) ES"0 BHeOID 
a (861) 16S | (€O'O) 810} (ZS°0) 9S FT| (HOO) Pr'E | (T8°P) 68T | (60°O) LL’E| (TO'O) ZS'T | (€0'0) 067 | (10°0) 9S°0 ladivy, souryD 
< (CET) T9V| (€O'O) STO} (160) I6°9T | (POO) TZ'E | (86'8) O9E| (SOO) ZE'H} (TOO) 971 | (ZO'O) LETZ| (100) TSO sy 
s (Lv) €€S| (€O'O) ZI} (ILO) ZEST | (POO) TLE} (ST'8) O8E| (SOO) S6'E | (ZO'O) 6ZT | (€0'0) ZOE | (100) LS‘0 (seqend) epeurD 
a. (vO'E) 81S | (€O'O) P70} (THO) TS IT} (€0'O) PLE} (9E'9) STE} (90°) 19°E| (ZO'O) LET | (€0'0) OTE | (100) 9S°0 (olreUQ) epeuRD 
< (Lv'7) LOS | (€0'O) €7°0} (L480) LIST | (OO) TSE} (S6'H) LOT} (SOO) LE} (ZOO) 8ZT| (€0'0) OEE | (ZOO) TSO (euoqgry) epeueD 
(Z9'€) 61h | (POO) 60} (EL'O) 9Z'ET | (HOO) OLE | (ZEB) ISZ| (600) 06'E | (ZOO) IT'Z| (€0'0) 66'T | (100) 8r'0 BUR ASIOG 
(p61) 6rS | (€0'O) 610} (TL'O) SILT | (€0'°0) E9'E | (BPH) LEE| (OOO) OO'T | (ZO'O) 8ZT| (€0'0) S67 | (10°0) 89°0 (ystwopq) wmnisjog 
(Sze) 9€V| (€O'O) P70} (L'0) GLIT| (LOO) 8r'E} (LEL) E7Z| CIT'O) OSE | (ZOO) SST | (HOO) 89°7 | (ZOO) 8E'0 urerye q 
(L8°S) €9h | (ZOO) 60°0} (HO'T) 8°77 | (LOO) 6L'Z | (SL'S) VET) (LOO) Thy} (ZOO) ETT | (€0'0) 107% Jo oygndoy ‘uelreqiezy 
(09°Z) 80S | (ZOO) 80°0} (89°0) PSIZ (SEE) 6EZ | (OOO) OO'T | (TO'O) LZT| (0'0) P67 | (100) 870 viysny 
(967) 91S | (€0'.O) E770} (60) LOOT] (€0'O) TLE} (SSL) 9VE| (POO) ZO'V| (TOO) ZZT | (ZOO) TEE | (ZOO) $9'0 Bye.ysny 
(LS‘€) TS} (100) 10°0} (9L'0) €6'SzZ| (SOO) OTE | (GO'T) TET] (800) EZ'F} (TO'O) OZ T | (€0'0) $8°z BIUULY 
JoUvULLIOJIOg oyeury, dxq poredaig sunyyepy |  dardyyeyy suey syoog | juowUsITy Ayunog 


p Opris TOT SSINLL ‘Sorqeuea Aoy JOJ suvoul [OAg[-ANuNOD pV IQUL 


149 


Appendix A: Detailed Country-Level Means for Key Variables 
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Appendix A: Detailed Country-Level Means for Key Variables 
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Appendix B 

Detailed Regression Estimates 
for the Single-level, Multilevel, 
and Classroom-mean Models 


We here present regression coefficients and standard errors for models analyzing the 
consistency of results across model types presented in Chap. 5 (see Fig. 5.1). 
Single-level, multilevel, and classroom-mean models for all education systems (also 
referred to as countries) where data was available are compared (Tables B.1, B.2, 
B.3, B.4, B.5, B.6, B.7, B.8, B.9, B.10, B.11, B.12, B.13, B.14 and B.15). Results 
are grouped by the consistency of results, significance levels, and direction of 
relationship between teacher quality indicators and student outcomes. Instances 
where relationships are not statistically significant in any of the three models are 
treated separately for each predictor. 
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Table B.1 Countries where the relationship of a given indicator of teacher quality with student 
achievement was statistically significant and in the same direction for all three statistical models 


Grade/year | Country Predictor | SLM MLM CLM 

G4Y07 Algeria Exp 1.6 (0.8) 1.3 (0.6) 1.8 (0.8) 
G4Y07 Hungary Mathtime 0.5 (0.2) 0.4 (0.2) 0.3 (0.1) 
G4Y07 Tran Exp 2.0 (0.5) 2.5 (0.3) 1.5 (0.6) 
G4Y07 Tran Alignment | —76.3 (32.6) | —65.6 (25.0) | —83.0 (34.0) 
G4Y07 Morocco Mathprep 20.2 (7.1) 13.9 (4.4) 20.0 (8.9) 
G4Y07 Tunisia Exp 3.1 (0.6) 2.8 (0.5) 2.3 (0.7) 
G4Y11 Bahrain Mathtime 0.1 (0.0) 0.1 (0.0) 0.1 (0.0) 
G4Y11 Germany Prepared 8.7 (3.8) 10.3 (3.2) 12.3 (5.8) 
G4Y11 Tran Exp 1.6 (0.4) 2.1 (0.4) 1.1 (0.5) 
G4Y11 United Arab Mathtime 0.1 (0.0) 0.1 (0.0) 0.1 (0.0) 

Emirates 
G4Y11 United States Prepared 12.1 (4.4) 12.7 (5.7) 23.7 (10.9) 
G4Y11 Northern Ireland | Alignment | —48.5 (22.6) | —26.6 (12.4) | —59.4 (27.3) 
G4Y11 United Arab Mathtime 0.1 (0.0) 0.1 (0.0) 0.1 (0.1) 
Emirates (Dubai) 

G4Y15 Canada Mathtime 0.1 (0.0) 0.1 (0.0) 0.1 (0.0) 
G4Y15 Chinese Taipei Mathtime 0.1 (0.0) 0.1 (0.0) 0.1 (0.0) 
G4Y15 France Alignment 38.9 (19.4) 55.3 (19.6) 82.1 (26.1) 
G4Y15 Georgia Alignment | —102.1 (47.5) | —107.3 (35) —131.4 (63.2) 
G4Y15 Korea Exp 0.7 (0.2) 0.5 (0.1) 0.4 (0.2) 
G4Y15 Morocco Exp —1.1 (0.4) —1.0 (0.3) —-1.0 (0.4) 
G4Y15 Portugal Alignment | —44.0 (13.6) | —32.0 (13.1) | —42.8 (15.4) 
G4Y15 Turkey Exp 1.9 (0.4) 1.7 (0.4) 0.9 (0.4) 
G8Y03 Chinese Taipei Prepared 24.4 (9.9) 8.3 (2.3) 16 (7.6) 
G8Y03 Jordan Mathtime 0.8 (0.3) 0.8 (0.2) 0.7 (0.2) 
G8Y03 Jordan Alignment 80.0 (38.0) 87.5 (32.5) 70.2 (34.3) 
G8Y03 Romania Prepared 35.3 (12.5) 20.6 (7.5) 42 (13.2) 
G8Y03 Singapore Alignment | 255.5 (46.1) | 314.1 (81.5) 149.4 (35.1) 
G8Y03 Macedonia Mathtime 0.5 (0.2) 0.4 (0.1) 0.4 (0.2) 
G8Y07 Australia Mathtime —0.2 (0.1) —0.2 (0.1) —0.3 (0.1) 
G8Y07 Botswana Exp 1.5 (0.7) 1.9 (0.5) 1.6 (0.6) 
G8Y07 Cyprus Alignment 64.9 (27.1) 66.8 (26.8) 61 (26.8) 
G8Y07 Palestinian Exp 1.5 (0.4) 1.6 (0.4) 1.7 (0.5) 
G8Y07 Hungary Mathtime 0.4 (0.1) 0.5 (0.2) 0.4 (0.1) 
G8Y07 Indonesia Exp 1.6 (0.6) 2.1 (0.4) 1.5 (0.5) 
G8Y07 Indonesia Alignment 188.9 (47.7) | 201.3 (38.3) 158.7 (40.6) 
G8Y07 Israel Exp 1.6 (0.6) 1.1 (0.5) 1.5 (0.6) 
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Grade/year | Country Predictor | SLM MLM CLM 

G8Y07 Japan Alignment | —140.3 (37.8) | —65.1 (23.6) | —124.2 (39.3) 
G8Y07 Jordan Mathtime 0.3 (0.1) 0.2 (0.1) 0.4 (0.1) 
G8Y07 Singapore Mathtime —0.6 (0.1) —0.6 (0.2) —0.2 (0.1) 
G8Y07 Turkey Mathtime 2.0 (0.5) 2.1 (0.5) 1.4 (0.3) 
G8Y07 Ukraine Mathtime 0.2 (0.1) 0.3 (0.1) 0.2 (0.1) 
G8Y07 Egypt Exp 1.8 (0.6) 1.7 (0.5) 2.0 (0.6) 
G8Y07 United States Alignment | —83.8 (14.7) | -164.5 (17.3) | —46.9 (14.4) 
G8Y07 Scotland Alignment | —323.8 (42.1) | -372.3 (54.1) | -222.6 (30.9) 
G8Y11 Botswana Prepared 19.6 (5.7) 21.6 (5.1) 11.8 (5.6) 
G8Y11 Chinese Taipei Mathprep 7.1 (3.0) 14.3 (4.7) 7.0 (2.7) 
G8Y11 Finland Mathprep 5.9 (2.8) 13.8 (3.2) 11.9 (3.7) 
G8Y11 Ghana Mathprep —7.6 (3.6) —6.5 (2.3) —10.5 (3.6) 
G8Y11 Indonesia Exp 1.3 (0.5) 1.8 (0.4) 1.4 (0.5) 
G8Y11 Israel Exp 1.7 (0.4) 0.8 (0.4) 1.1 (0.4) 
G8Y11 Israel Mathtime 0.2 (0.1) 0.4 (0.1) 0.2 (0.1) 
G8Y11 Italy Mathprep —12.4 (4.9) —10.4 (3.8) —10.7 (5.1) 
G8Y11 Japan Mathprep 5.1 (2.3) 6.0(3.0) 4.2 (2.1) 
G8Y11 Japan Mathtime 0.4 (0.1) 0.4 (0.1) 0.4 (0.1) 
G8Y11 Korea Exp 0.8 (0.2) 1.1 (0.3) 0.8 (0.3) 
G8Y11 Morocco Mathtime 0.4 (0.1) 0.5 (0.1) 0.2 (0.1) 
G8Y11 Oman Exp 1.8 (0.5) 1.0 (0.4) 1.5 (0.4) 
G8Y11 Oman Prepared 10.9 (4.9) 12.3 (3.5) 11.6 (4.8) 
G8Y11 Norway Alignment 43.7 (18.7) 36.1 (15.8) 48.4 (19.2) 
G8Y11 Singapore Mathtime —0.2 (0.0) —0.2 (0.1) —0.1 (0.0) 
G8Y11 Singapore Alignment | —93.4 (31.1) | -128.7 (52) —53.9 (26.1) 
G8Y11 Slovenia Alignment 40.1 (20.1) 74.9 (24.4) 49.8 (20.5) 
G8Y11 Syria Mathtime —0.1 (0.1) —0.1 (0.0) —0.1 (0.1) 
G8Y11 Tunisia Exp 1.4 (0.3) 1.6 (0.2) 0.5 (0.2) 
G8Y11 England Alignment 146.2 (36.6) 95.1 (38.1) 98.1 (28.4) 
G8Y15 Canada Mathtime —0.1 (0.0) —0.1 (0.0) —0.1 (0.0) 
G8Y15 Chinese Taipei Mathtime 0.2 (0.1) 0.3 (0.1) 0.2 (0.1) 
G8Y15 Georgia Mathtime 0.1 (0.0) 0.1 (0.0) 0.1 (0.0) 
G8Y15 Hungary Mathtime 0.2 (0.1) 0.2 (0.1) 0.1 (0.1) 
G8Y15 Israel Exp 1.2 (0.4) 0.9 (0.3) 1.0 (0.3) 
G8Y15 Israel Mathtime 0.5 (0.1) 0.5 (0.1) 0.4 (0.1) 
G8Y15 Japan Mathtime 0.4 (0.1) 0.1 (0.1) 0.2 (0.1) 
G8Y15 Japan Alignment | —128.9 (25.8) | —72.5 (30.8) | —100.6 (35.9) 
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Table B.1 (continued) 


Grade/year | Country Predictor |SLM MLM CLM 

G8Y15 Jordan Mathtime 0.2 (0.1) 0.2 (0.0) 0.2 (0.1) 
G8Y15 Singapore Mathtime —0.7 (0.1) —0.6 (0.1) —0.3 (0.1) 
G8Y15 Singapore Alignment | —146.6 (22.1) | —131.8 (30.4) | —43.8 (20.5) 
G8Y15 Thailand Alignment | 104.7 (27.6) | 103.4 (26.4) 46.8 (23) 
G8Y15 United Arab Exp —1.0 (0.4) —0.9 (0.4) —1.0 (0.4) 

Emirates 
G8Y15 Egypt Mathtime 0.2 (0.1) 0.2 (0.1) 0.2 (0.1) 


Notes Standard errors in parentheses. Grade/year indicates grade level (where G4 and G8 are grade 
four and eight, respectively) and year (where e.g., YO7 is the year 2007), resulting in a composite 
abbreviation such as G4Y07 (which indicates Grade 4, 2007 cycle of TIMSS data). SLM = 
single-level model, MLM = multilevel model, CLM = classroom-level model, Alignment = 
proportion of mathematics topics covered compared with national expectations, Mathprep = index 
(1-5) of teacher education to teach mathematics, Mathtime = mean number of minutes spent 
teaching mathematics per week, Prepared = index (1-4) of self-efficacy to teach mathematics, Exp 
= experience teaching in years 


Table B.2 Countries where the relationship of a given indicator of teacher quality with student 
achievement was statistically significant but in the different directions across the three statistical 
models 


Grade/year Country Predictor SLM MLM CLM 
G8Y07 Malta Alignment 160.3 (7.4) —33.0 (13.1) 103.5 (25.9) 


Notes Standard errors in parentheses. Grade/year indicates grade level (where G4 and G8 are grade 
four and eight, respectively) and year (where e.g., YO7 is the year 2007), resulting in a composite 
abbreviation such as G4Y07 (which indicates Grade 4, 2007 cycle of TIMSS data). SLM = single 
level model, MLM = multilevel model, CLM = classroom-level model, Alignment = proportion of 
mathematics topics covered compared with national expectations, Mathprep = index (1-5) of 
teacher education to teach mathematics, Mathtime = mean number of minutes spent teaching 
mathematics per week, Prepared = index (1-4) of self-efficacy to teach mathematics, Exp = 
experience teaching in years 
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Table B.3 Countries where the relationship of a given indicator of teacher quality with student 
achievement was statistically significant and in the same direction for the single-level and 
multilevel models 


Grade/year | Country Predictor SLM MLM CLM 
G4Y03 Tran Exp 1.2 (0.5) 1.5 (0.5) —0.6 (1.2) 
G4Y07 Austria Alignment 29.1 (14.1) 31.5 (12.2) | 23.9 (18.9) 
G4Y07 El Salvador Mathprep 8.3 (2.8) 8.8 (2.5) 3.7 (3.9) 
G4Y07 El Salvador Prepared 20.0 (7) 8.8 (2.5) 13.6 (7.6) 
G4Y07 Germany Mathprep 11.8 (5.5) 13.1 (5.6) 16.0 (16.7) 
G4Y07 Lithuania Exp —0.8 (0.3) —0.8 (0.2) —0.7 (0.4) 
G4Y07 Tunisia Mathprep 8.2 (3.1) 7.0 (2.9) 7.0 (5.0) 
G4Y07 England Exp 0.6 (0.2) 0.4 (0.2) 0.3 (0.2) 
G4Y07 United States Alignment | —67.2 (23.9) | —79.2 (38.5) | -37.5 (24.2) 
(Massachusetts) 
G4Y11 Chinese Taipei Mathprep 3.9 (1.8) 3.7 (1.1) 2.4 (1.7) 
G4Y11 Denmark Alignment 33.8 (16.3) 36 (10.2) 23.7 (15.3) 
G4Y11 Finland Prepared 9.9 (4.2) 8.0 (3.2) 6.6 (6.0) 
G4Y11 Germany Alignment 25.0 (12.0) 32.1 (11.7) 33.5 (20.0) 
G4Y11 Hong Kong Exp 0.8 (0.4) 1.3 (0.3) 0.7 (0.4) 
G4Y11 Japan Mathtime —0.2 (0.1) -0.1 (0.0) —0.1 (0.1) 
G4Y11 Malta Exp 0.5 (0.2) 0.6 (0.2) 0.7 (0.4) 
G4Y11 Portugal Exp 0.7 (0.3) 0.7 (0.3) 0.8 (0.7) 
G4Y11 Qatar Alignment | —79.8 (34.9) | -—76.2 (29.2) | —33.1 (35.3) 
G4Y11 United Arab Emirates Mathtime 0.2 (0.1) 0.2 (0.1) 0.1 (0.1) 
(Abu Dhabi) 

G4yY15 Hungary Alignment | —48.3 (22.4) | —51.0 (21.2) | —53.8 (27.7) 
G4yY15 Qatar Alignment | —59.7 (25.4) | —57.8 (17.7) | -17.2 (23.6) 
G4Y15 Singapore Prepared 15.2 (4.8) 13.8 (6.7) 8.1 (4.2) 
G4yY15 England Prepared 14.7 (5.7) 19.5 (7.1) 6.4 (6.4) 
G8Y03 Chile Mathprep 8.0 (3.1) 8.2 (2.7) 2.3 (2.4) 
G8Y03 Chile Prepared 34.5 (9.6) 8.2 (2.7) 11 (7.8) 
G8 Y03 Chinese Taipei Mathprep 6.8 (2.9) 8.3 (2.3) 2.5 (2.2) 
G8Y03 Chinese Taipei Mathtime 0.2 (0.1) 0.2 (0.1) 0.0 (0.1) 
G8 Y03 Tran Exp 1.2 (0.4) 1.3 (0.3) 0.3 (0.4) 
G8 Y03 Egypt Alignment 94.6 (38.6) 78 (31.9) | 60.8 (54.3) 
G8 Y03 Canada (Quebec) Mathtime —0.2 (0.1) —0.1 (0.0) —0.1 (0.0) 
G8Y07 Australia Mathprep 7.9 (2.9) 5.8 (2.8) 3.3 (2.6) 
G8Y07 Chinese Taipei Mathtime 0.2 (0.1) 0.2 (0.1) 0.1 (0.1) 
G8Y07 Colombia Mathprep 9.0 (4.1) 9.8 (2.9) 3.1 (4.2) 
G8Y07 Ghana Exp 1.6 (0.6) 1.3 (0.5) 0.5 (0.7) 
G8Y07 Tran Exp 0.8 (0.4) 1.5 (0.3) —0.4 (0.5) 
G8Y07 Qatar Exp —0.9 (0.2) -0.7 (0.2) —0.6 (0.4) 
G8Y07 Tunisia Exp 1.0 (0.3) 1.0 (0.2) 0.5 (0.3) 
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Grade/year | Country Predictor SLM MLM CLM 
G8Y07 Turkey Exp 1.5 (0.6) 2.2 (0.5) —0.3 (0.6) 
G8Y07 Ukraine Alignment | —116.6 (44.8) | —136.0 (50.2) | -73.9 (40.1) 
G8Y07 United States Alignment | —88.9 (32.7) | —159.0 (33.1) | —15.3 (32.5) 
(Massachusetts) 

G8Y11 Chile Mathprep 8.9 (3.1) 13.2 (2.7) 3.8 (2.7) 
G8Y11 Chile Alignment 57.6 (23.3) 70.7 (22.4) 5.9 (23.1) 
G8Y11 Tran Exp 1.7 (0.5) 2.6 (0.4) 0.5 (0.5) 
G8Y11 Israel Mathprep 11.1 (4.3) 7.3 (3.6) 6.5 (4.1) 
G8Y11 Italy Alignment 78.8 (28) 91.5 (22.7) | 58.4 (30.7) 
G8Y11 Malaysia Mathprep —8.1 (3.7) —8.0 (3.4) —4.3 (2.8) 
G8Y11 South Africa Prepared 23.2 (5.1) 40.9 (11.4) 9.7 (5.4) 
G8Y11 Thailand Exp 0.9 (0.3) 1.0 (0.3) 0.6 (0.4) 
G8Y11 Thailand Mathtime 0.2 (0.1) 0.1 (0.0) 0.1 (0.0) 
G8Y11 Canada (Quebec) Exp 0.8 (0.3) 2.9 (0.9) 0.4 (0.3) 
G8Y15 Chile Mathprep 9.0 (3.4) 11.1 (3.4) 5.1 (3.2) 
G8Y15 Chinese Taipei Exp 0.6 (0.3) 0.9 (0.2) 0.4 (0.3) 
G8Y15 Tran Exp 1.7 (0.5) 2.1 (0.3) —0.4 (0.6) 
G8Y15 Tran Mathprep 9.1 (3.6) 6.5 (3.1) 6.1 (3.1) 
G8Y15 Malta Exp —0.8 (0.1) —0.4 (0.2) —0.1 (0.4) 
G8Y15 Malta Alignment 128.8 (7.1) 113 (7.5) 24.3 (22.8) 
G8Y15 Malta Prepared 17.5 (2.3) 18.8 (2.6) 1.4 (8.3) 
G8Y15 Slovenia Prepared 6.1 (2.6) 6.8 (2.3) 4.5 (3.0) 
G8Y15 South Africa Mathtime —0.2 (0.1) —0.1 (0.0) —0.1 (0.0) 
G8Y15 Turkey Exp 1.2 (0.5) 1.8 (0.4) —0.4 (0.5) 
G8Y15 England Prepared 17.6 (6.4) 30.3 (14.8) 6.4 (5.4) 


Notes Standard errors in parentheses. Grade/year indicates grade level (where G4 and G8 are grade four and 
eight, respectively) and year (where e.g., YO7 is the year 2007), resulting in a composite abbreviation such 
as G4Y07 (which indicates Grade 4, 2007 cycle of TIMSS data). SLM = single level model, MLM = 
multilevel model, CLM = classroom-level model, Alignment = proportion of mathematics topics covered 
compared with national expectations, Mathprep = index (1-5) of teacher education to teach mathematics, 
Mathtime = mean number of minutes spent teaching mathematics per week, Prepared = index (1-4) of 
self-efficacy to teach mathematics, Exp = experience teaching in years 


Table B.4 Countries where the relationship of a given indicator of teacher quality with student 
achievement was statistically significant and in the opposite direction for the single-level and 
multilevel models 


Grade/year Country Predictor SLM MLM CLM 
G8Y07 Jordan Prepared 32.1 (10.0) —9.9 (5.0) 23.3 (14.5) 
G8Y07 Tunisia Prepared 8.7 (4.4) —5.5 (2.1) 5.5 (3.8) 


Notes Standard errors in parentheses. Grade/year indicates grade level (where G4 and G8 are 
grade four and eight, respectively) and year (where e.g., YO7 is the year 2007), resulting in a 
composite abbreviation such as G4Y07 (which indicates Grade 4, 2007 cycle of TIMSS data). 
SLM = single level model, MLM = multilevel model, CLM = classroom-level model, Prepared = 
index (1-4) of self-efficacy to teach mathematics 
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Table B.5 Countries where the relationship of a given indicator of teacher quality with student 
achievement was statistically significant and in the same direction for the multilevel and 
classroom-mean models 


Grade/year | Country Predictor | SLM MLM CLM 

G4Y03 Chinese Taipei Exp 0.3 (0.2) 0.3 (0.1) 0.3 (0.2) 
G4Y07 Morocco Mathtime -0.3 (0.1) —0.2 (0.1) —-0.3 (0.1) 
G4Y07 Morocco Alignment | -119.8 (67.2) | -101.8 (33.2) | —275.6 (74.7) 
G4Y07 Canada (British Mathtime —0.1 (0.0) —0.1 (0.0) —0.1 (0.0) 

Columbia) 
G4Y11 Germany Mathprep 2.7 (1.7) 3.0 (1.4) 5.6 (2.8) 
G4Y11 Hungary Mathtime 0.1 (0.0) 0.1 (0.0) 0.1 (0.0) 
G4Y11 Morocco Exp —1.1 (0.6) —1.2 (0.3) —-1.8 (0.9) 
G4Y11 Morocco Mathtime -0.1 (0.1) —0.2 (0.1) —0.2 (0.1) 
G4Y11 Qatar Exp 1.3 (0.7) 1.1 (0.5) 1.5 (0.7) 
G4Y15 Canada Prepared 14.9 (8.3) 8.3 (4.1) 22.6 (9.8) 
G4Y15 Chile Mathtime 0.1 (0.0) 0.1 (0.0) 0.1 (0.0) 
G4Y15 Germany Mathprep 5.9 3.1) 6.2 (1.7) 9.9 (5.0) 
G8Y03 Jordan Exp 0.8 (0.4) 0.8 (0.3) 1.0 (0.5) 
G8Y03 New Zealand Alignment 81.2 (41.6) 107.1 (41.8) 88.5 (41.9) 
G8Y11 Australia Exp 1.0 (0.5) 1.0 (0.3) 1.0 (0.4) 
G8Y11 Palestinian Prepared 10.6 (5.4) 7.9 (3.4) 12.2 (6.1) 
G8Y11 Ukraine Alignment 71.4 (44.2) 73.3 (25) 117.1 (51.3) 
G8Y15 Australia Mathprep 4.0 (2.5) 12.1 (5.9) 5.7 (1.9) 
G8Y15 Botswana Alignment 31.1 (17.3) 46.0 (15.1) 36.0 (14.6) 
G8Y15 Korea Exp 0.4 (0.2) 1.0 (0.4) 0.8 (0.3) 
G8Y15 Sweden Alignment 24.8 (20.1) 54.9 (22.4) 46.5 (21.1) 
G8Y15 Thailand Prepared -10.7 (6.1) -16.5 (5.4) —-10.5 (5.2) 
G8Y15 United Arab Emirates Exp -0.9 (0.5) —-1.4 (0.5) —1.3 (0.5) 
(Dubai) 


Notes Standard errors in parentheses. Grade/year indicates grade level (where G4 and G8 are grade four 
and eight, respectively) and year (where e.g., Y07 is the year 2007), resulting in a composite abbreviation 
such as G4Y07 (which indicates Grade 4, 2007 cycle of TIMSS data). SLM = single level model, MLM = 
multilevel model, CLM = classroom-level model, Alignment = proportion of mathematics topics covered 
compared with national expectations, Mathprep = index (1-5) of teacher education to teach mathematics, 
Mathtime = mean number of minutes spent teaching mathematics per week, Prepared = index (1-4) of 
self-efficacy to teach mathematics, Exp = experience teaching in years 


Table B.6 Countries where the relationship of a given indicator of teacher quality with student 
achievement was statistically significant and in the opposite direction for the multilevel and 
classroom-mean models 


Grade/year Country Predictor SLM MLM CLM 
G8Y15 Tran Mathtime 0.1 (0.0) 0.1 (0.0) —0.1 (0.0) 


Notes Grade/year indicates grade level (where G4 and G8 are grade four and eight, respectively) 
and year (where e.g., YO7 is the year 2007), resulting in a composite abbreviation such as G4Y07 
(which indicates Grade 4, 2007 cycle of TIMSS data). SLM = single level model, MLM = 
multilevel model, CLM = classroom-level model, Mathtime = mean number of minutes spent 
teaching mathematics per week 
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Table B.7 Countries where the relationship of a given indicator of teacher quality with student 
achievement was statistically significant and in the same direction for single level and classroom 


mean models 


Grade/year | Country Predictor | SLM MLM CLM 
G4Y03 Canada (Quebec) Prepared 19.9 (7.5) 2.7 (2.2) 22.8 (8.7) 
G4Y07 Australia Prepared 18.9 (6) 1.1 (5.6) 23.3 (6.7) 
G4Y07 Colombia Prepared 32.5 (7.2) —1.1 (2.7) 34.4 (9.8) 
G4Y07 Czech Republic Prepared 7.5 (3.5) 2.5 (1.9) 6.0 (3.0) 
G4Y07 Denmark Mathprep 4.9 (2.2) 2.5 (2.0) 5.1 (2.6) 
G4Y07 New Zealand Prepared 8.9 (3.8) —4.0 (4.2) 12.0 (3.7) 
G4Y07 United States Prepared 24.8 (5.1) 3.4 (3.1) 21.4 (5.1) 
G4Y07 Scotland Prepared 25.9 (10.4) 1.6 (3.8) 29.9 (10.7) 
G4Y07 Canada (British Prepared 15.1 (5.4) 0.5 (3.4) 15.1 (5.8) 
Columbia) 
G4yY11 Netherlands Alignment | 32.7 (14.6) 29.8 (17.4) 31.6 (15.3) 
G4Y15 Bahrain Mathprep 6.3 (2.4) 5.8 (3.6) 7A (2.6) 
G4Y15 Kuwait Mathtime 0.2 (0.1) 0.1 (0.1) 0.2 (0.1) 
G4Y15 New Zealand Prepared 20.6 (5.1) 8.0 (5.0) 16.9 (5.6) 
G4Y15 Poland Mathtime 0.3 (0.2) 0.2 (0.3) 0.4 (0.1) 
G4Y15 United Arab Mathprep | —10.4 (4.1) 0.3 (2.1) —11.7 (4.8) 
Emirates 
G4Y15 United Arab Prepared 24.6 (6.8) 9.8 (6.6) 20.0 (9.1) 
Emirates 
G4Y15 United States Exp 0.6 (0.3) 0.3 (0.3) 0.6 (0.2) 
G4Y15 Argentina (Buenos | Exp 1.3 (0.5) 1.1 (0.7) 1.2 (0.6) 
Aires) 
G8Y03 Bahrain Exp 0.9 (0.3) 0.2 (0.4) 1.0 (0.3) 
G8Y03 Bahrain Mathprep 6.9 (2.4) 3.9 (2.8) 8.9 (2.3) 
G8Y03 Hungary Alignment | 83.9 (35.9) 61.9 (40.2) 82.9 (37.8) 
G8Y03 Israel Exp 1.2 (0.5) 0.9 (0.5) 1.1 (0.5) 
G8Y03 Jordan Prepared 19.5 (8.1) 0.4 (4.1) 21.8 (7.9) 
G8Y03 New Zealand Mathprep 7.9 (3.4) 2.2 (3.2) 9 (2.8) 
G8Y03 Singapore Mathtime —0.3 (0.1) —0.2 (0.1) —0.2 (0.1) 
G8Y03 Sweden Alignment | 67.2 (28.5) 43.3 (51.9) 73.1 (33.7) 
G8Y07 Chinese Taipei Prepared 11.4 G.8) 1.3 (2.5) 8.4 (3.4) 
G8Y07 Czech Republic Mathtime 0.3 (0.1) 0.2 (0.1) 0.3 (0.1) 
G8Y07 Ghana Prepared 45.1 (13.3) 0.1 (2.2) 49.0 (14.2) 
G8Y07 Hong Kong Alignment 202 (100.4) | 44.0 (126.8) | 194.3 (70.5) 
G8Y07 Qatar Mathtime 0.2 (0.0) 0.0 (0.1) 0.1 (0.1) 
G8Y07 Qatar Prepared 15.5 (2.6) 0.7 (1.7) 13.2 (5.9) 
G8Y07 Singapore Alignment | 103.6 (34.8) 102.9 (57.7) 79 (21.6) 
G8Y07 Singapore Prepared 17.5 (7.6) —3.3 (4.1) 12.5 (5.5) 
G8Y07 United States Mathprep 4.4 (2.0) 3.1 (3.4) 5.9 (2.0) 
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Table B.7 (continued) 
Grade/year | Country Predictor | SLM MLM CLM 
G8Y07 United States Prepared 26.7 (10.7) 3.1 (3.4) 28 (10.3) 
G8Y07 Scotland Mathtime 0.2 (0.1) 0.2 (0.2) 0.2 (0.1) 
G8Y11 Ghana Exp 1.7 (0.7) 0.9 (0.6) 1.7 (0.8) 
G8Y11 Hungary Mathprep 11.0 (4.2) 5.6 (4.2) 7.2 (3.6) 
G8Y11 Indonesia Alignment | 128.8 (38.5) 70.2 (37.0) 89.7 (39.0) 
G8Y11 New Zealand Mathtime 0.3 (0.1) 0.4 (0.2) 0.3 (0.1) 
G8Y11 United States Alignment | —86.1 (13.1) | -54.7 (31.8) | —47.5 (13.7) 
G8Y11 United Arab Alignment | 44.8 (22.6) | —10.6 (47.8) 54.8 (26.8) 
Emirates (Dubai) 
G8Y15 Bahrain Mathprep 6.3 (2.7) 5.3 (3.0) 7.7 (2.9) 
G8Y15 Canada Mathprep 7.3 (2.0) 1.9 (2.4) 7.5 (2.6) 
G8Y15 Canada Prepared -7.1 (3.4) —5.2 (4.4) —8.0 (3.4) 
G8Y15 Kazakhstan Mathprep 20.6 (7.9) 7.3 (9.8) 20.2 (7.5) 
G8Y15 Thailand Mathprep —9.6 (3.4) —5.9 (3.4) —9.0 (3.8) 
G8Y15 United Arab Mathprep 12.8 (4.6) —3.1 (3.9) 12.8 (3.7) 
Emirates 
G8Y15 United Arab Mathtime —0.1 (0.0) 0.0 (0.1) —0.1 (0.0) 
Emirates 
G8Y15 United Arab Prepared 20.9 (5.7) 0.4 (4.8) 11.5 (5.4) 
Emirates 
G8Y15 Canada (Quebec) Mathprep 7.1 (3.0) 4.3 (5.2) 7A (2.5) 


Notes Standard errors in parentheses. Grade/year indicates grade level (where G4 and G8 are grade 
four and eight, respectively) and year (where e.g., YO7 is the year 2007), resulting in a composite 
abbreviation such as G4Y07 (which indicates Grade 4, 2007 cycle of TIMSS data). SLM = single 
level model, MLM = multilevel model, CLM = classroom-level model, Alignment = proportion of 
mathematics topics covered compared with national expectations, Mathprep = index (1-5) of 
teacher education to teach mathematics, Mathtime = mean number of minutes spent teaching 
mathematics per week, Prepared = index (1-4) of self-efficacy to teach mathematics, Exp = 
experience teaching in years 
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Table B.8 Countries where the relationship of a given indicator of teacher quality with student 
achievement was statistically significant for only the multilevel model 


Grade/year | Country Predictor SLM MLM CLM 
G4Y03 Canada (Quebec) Mathtime 0.0 (0.0) 0.0 (0.0) 0.0 (0.0) 
G4Y07 Algeria Mathprep 8.9 (5.8) 6.5 (3.1) 7.9 (6.3) 
G4Y07 Algeria Prepared —0.6 (9.4) 6.5 (3.1) 5.7 (11.7) 
G4Y07 Chinese Taipei Exp 0.4 (0.2) 0.5 (0.2) 0 (0.2) 
G4Y07 El Salvador Alignment | 37.1 (31.1) 54.6 (24.7) 16.8 (46) 
G4Y07 Georgia Mathprep —0.6 (5.2) —-7.0 (3.0) —-0.1 (7.1) 
G4Y07 Georgia Prepared 5.1 (11.3) -7.0 (3.0) 5.2 (16.4) 
G4Y07 Germany Prepared 1.6 (4.5) 13.1 (5.6) —1.4 (5.7) 
G4Y07 Hong Kong Mathtime 0.1 (0.0) 0.2 (0.1) 0.0 (0.0) 
G4Y07 Tran Mathprep 3.1 (1.7) 4.9 (1.4) 1.5 (2.1) 
G4Y07 Tran Prepared —-47 (4.7) 4.9 (1.4) | -10.4 (6.1) 
G4Y07 Morocco Prepared 20.2 (10.6) 13.9 (4.4) 14.5 (13.3) 
G4Y07 Netherlands Mathtime —-0.1 (0.1) —0.1 (0.0) 0.0 (0.1) 
G4Y07 Singapore Alignment | —27.4 (23.4) | —64.3 (32.7) | —25.0 (18) 
G4Y07 Tunisia Prepared -1.3 (6.1) 7.0 (2.9) —2.6 (7.7) 
G4Y07 United States Mathtime 0.0 (0.0) 0.0 (0.0) 0.0 (0.0) 
G4Y07 Canada (Ontario) Alignment | —18.4 (20.9) | —56.3 (28) —26.7 (20.7) 
G4Y07 United States Mathtime —0.1 (0.0) —0.1 (0.0) 0.0 (0.0) 
(Massachusetts) 

G4Y11 Botswana Exp 1.0 (0.6) 0.8 (0.4) 0.9 (0.6) 
G4Y11 Czech Republic Mathprep 5.4 (2.9) 5.5 (2.5) 6.9 (4.8) 
G4Y11 Honduras Mathprep 3.2 (4.3) 4.6 (2.1) 2.7 (5.2) 
G4Y11 Hong Kong Mathprep 6.9 (3.9) 7.8 (3.8) 5.0 (4.2) 
G4Y11 Tran Mathtime 0.1 (0.0) 0.1 (0.0) 0.0 (0.0) 
G4Y11 Japan Alignment | —19.5 (13.3) | —25.6 (11.0) | —4.4 (16.7) 
G4Y11 Korea Mathtime 0.0 (0.0) 0.0 (0.0) 0.0 (0.0) 
G4Y11 Morocco Mathprep 4.3 (6.2) 11.7 (4.8) 0.5 (6.4) 
G4Y11 Qatar Prepared 5.7 (14.5) 22.8 (9.8) 74 (13.4) 
G4Y11 Singapore Exp —0.5 (0.3) —1.1 (0.5) —0.2 (0.2) 
G4Y11 Thailand Prepared 12.6 (6.6) 8.2 (3.3) 10.2 (6.8) 
G4yY11 Tunisia Exp 0.9 (0.5) 0.9 (0.4) 0.1 (0.6) 
G4Y11 Canada (Quebec) Alignment 0.4 (9.5) 25.7 (11.9) 17 (11.1) 
G4yY15 Australia Exp 0.0 (0.2) 0.4 (0.2) 0.1 (0.2) 
G4yY15 Australia Mathprep 4.3 (4.6) 7.7 (3.3) 9.4 (5.3) 
G4yY15 Bahrain Exp 0.6 (0.3) 0.9 (0.4) 0.6 (0.4) 
G4yY15 Chile Mathprep 5.6 (6.8) 9.6 (4.7) 3.7 (5.4) 
G4yY15 Chile Prepared —10.3 (13.6) | —19.6 (5.4) | -11.5 (11.6) 
G4yY15 Cyprus Alignment | 26.4 (15.8) 27.2 (13.4) | 25.1 (14.8) 
G4yY15 Georgia Prepared —13.0 (8.7) —12.8 (5.7) | -14.4 (11.0) 
G4yY15 Germany Exp 0.2 (0.1) 0.2 (0.1) 0.3 (0.2) 
G4yY15 Germany Alignment 19.9 (12.4) 22.9 (11.2) | 24.5 (13.1) 
G4yY15 Hong Kong Alignment 14.2 (29.8) 135.7 (54.4) 18.6 (29.8) 
G4Y15 Indonesia Prepared 11.0 (6.4) 13.3 (5.0) 6.0 (6.6) 
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Table B.8 (continued) 

Grade/year | Country Predictor SLM MLM CLM 
G4yY15 Tran Alignment | 20.9 (26.7) 33.7 (16.4) | —4.5 (31) 
G4yY15 Japan Mathtime -0.1 (0.1) -0.1 (0.0) —0.1 (0.1) 
G4Y15 Lithuania Prepared 4.7 (6.3) 8.7 (3.8) 3.2 (5.3) 
G4yY15 Oman Prepared 13.6 (7.0) 10.4 (4.6) 11.5 (7.0) 
G4yY15 Poland Alignment | —21.0 (17.9) | —32.4 (14.2) | —8.1 (17) 
G4yY15 Qatar Prepared 9.2 (9.9) 15.2 (5.5) 13.2 (8.7) 
G4yY15 Serbia Prepared 13.5 (8.3) 12 (5.2) 19.6 (10.4) 
G4Y15 Singapore Exp —0.5 (0.3) —1.1 (0.5) —0.3 (0.2) 
G4yY15 Sweden Mathtime -0.1 (0.1) —0.1 (0.0) 0.0 (0.1) 
G4yY15 Turkey Mathprep 5.7 (4.2) 12.3 (3.9) 2.4 (4.0) 
G4yY15 Turkey Prepared 9.9 (6.4) 14.6 (6.4) 1.1 (6.8) 
G4Y15 United States Mathprep -1.7 B.6) —8.4 (4.0) —2.4 (3.0) 
G4yY15 Northern Ireland Mathtime 0.1 (0.0) 0.1 (0.0) 0.1 (0.1) 
G8Y03 Botswana Exp 1.9 (1.0) 2.3 (0.7) 0.9 (0.5) 
G8Y03 Palestinian Exp 0.9 (0.5) 0.8 (0.4) 0.7 (0.5) 
G8Y03 Iran Mathtime 0.1 (0.1) 0.2 (0.1) 0.1 (0.1) 
G8 Y03 Lithuania Mathtime 0.1 (0.1) 0.4 (0.2) 0.0 (0.1) 
G8 Y03 Philippines Exp 1.7 (0.9) 1.5 (0.6) 1.2 (0.7) 
G8Y03 Romania Exp 0.2 (0.3) 0.7 (0.3) 0.0 (0.4) 
G8 Y03 Romania Mathprep 9.4 (9.2) 20.6 (7.5) 7.5 (11.1) 
G8Y03 Sweden Mathtime 0.0 (0.1) 0.2 (0.1) 0.1 (0.1) 
G8Y03 Egypt Exp 1.3 (1.1) 2.2 (0.9) 0.9 (1.4) 
G8Y03 Canada (Ontario) Alignment | 27.5 (30.4) 40 (16.6) 36.1 (25.4) 
G8Y03 Canada (Quebec) Mathprep 3.7 (3.7) 5.6 (2.2) 1.4 (2.8) 
G8 Y03 Canada (Quebec) Alignment 5.8 (32) 39.5 (19.2) 15.6 (23.2) 
G8Y03 Canada (Quebec) Prepared 5.5 (12.3) 5.6 (2.2) 7.3 (9.6) 
G8Y07 Australia Prepared —2.2 (8.8) 5.8 (2.8) 0.1 (8.0) 
G8Y07 Bosnia and Herzegovina Mathtime —-0.1 (0.2) -0.2 (0.1) 0.0 (0.1) 
G8Y07 Bulgaria Mathprep 15.7 (9.1) 15.0 (6.7) 12.8 (12.1) 
G8Y07 Bulgaria Prepared 11.8 (21.1) 15.0 (6.7) 20.8 (26.0) 
G8Y07 Colombia Prepared 10.6 (13.1) 9.8 (2.9) —2.6 (10.8) 
G8Y07 Cyprus Exp 0.4 (0.2) 0.4 (0.2) 0.3 (0.2) 
G8Y07 Hungary Exp 0.4 (0.3) 0.8 (0.3) 0.3 (0.3) 
G8Y07 Tran Mathtime 0.1 (0.1) 0.1 (0.0) 0.0 (0.1) 
G8Y07 Jordan Mathprep —5.8 (6.8) —-9.9 (5.0) —1.3 (7.3) 
G8Y07 Lebanon Mathprep —0.8 (4.2) —-3.1 (1.4) —1.0 (3.3) 
G8Y07 Lebanon Prepared 2.7 (10.9) —-3.1 (1.4) —1.4 (10.7) 
G8Y07 Sweden Mathtime 0.1 (0.1) 0.1 (0.1) 0.1 (0.1) 
G8Y07 Thailand Alignment | 64.6 (57.5) 84.5 (41.6) | 42.3 (47.5) 
G8Y07 Tunisia Mathprep —4,.2 (2.7) —5.5 (2.1) —3.5 (2.2) 
G8Y07 Turkey Mathprep 5.0 (4.9) 10.3 (4.8) —0.5 (4.9) 
G8Y07 Turkey Prepared 13.2 (8.1) 10.3 (4.8) 8.0 (7.0) 
G8Y07 Spain (Basque) Alignment | 47.4 (26.4) 62.5 (26.2) | 50.0 (34.2) 
G8Y11 Botswana Exp 1.3 (0.8) 1.5 (0.5) 0.9 (0.5) 
G8Y11 Chile Mathtime 0.0 (0.0) 0.1 (0.0) 0.0 (0.0) 
G8Y11 Chile Prepared 9.4 (7.5) 19 (5.2) 0.2 (5.9) 
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Grade/year | Country Predictor SLM MLM CLM 
G8Y11 Finland Exp 0.2 (0.2) 0.4 (0.2) 0.0 (0.2) 
G8Y11 Georgia Mathtime 0.2 (0.1) 0.2 (0.1) 0.1 (0.1) 
G8Y11 Palestinian Exp 0.6 (0.5) 0.7 (0.3) 0.4 (0.5) 
G8Y11 Palestinian Mathprep -7.8 (4.1) —6.8 (2.7) -7.6 (4.5) 
G8Y11 Hong Kong Exp —0.4 (0.6) —2.6 (1.3) —0.4 (0.5) 
G8Y11 Hong Kong Mathtime —-0.1 (0.1) -0.2 (0.1) 0.0 (0.1) 
G8Y11 Hungary Prepared 4.7 (5.0) 18.4 (5.4) 10.2 (7.0) 
G8Y11 Tran Mathtime 0.1 (0.1) 0.1 (0.0) 0.0 (0.0) 
G8Y11 Tran Prepared 13.1 (6.9) 15.6 (5.9) 12.4 (7.0) 
G8Y11 Jordan Mathtime 0.1 (0.0) 0.1 (0.0) 0.0 (0.1) 
G8Y11 Korea Mathtime 0.1 (0.1) 0.2 (0.1) 0.1 (0.1) 
G8Y11 Oman Alignment | —13.5 (26.1) | —42.2 (19.1) | —24.1 (23.1) 
G8Y11 New Zealand Exp 0.1 (0.3) —0.8 (0.4) 0.0 (0.2) 
G8Y11 New Zealand Mathprep 6.6 (3.7) 12.4 (4.6) 5.4 (3.0) 
G8Y11 Syria Exp 1.0 (0.6) 1.0 (0.4) 1.1 (0.6) 
G8Y11 Tunisia Mathtime 0.1 (0.1) 0.1 (0.0) 0.1 (0.1) 
G8Y11 Turkey Exp 0.8 (0.5) 1.2 (0.4) —0.6 (0.5) 
G8Y11 Ukraine Mathtime 0.2 (0.1) 0.3 (0.1) 0.1 (0.1) 
G8Y11 United States Mathtime 0.0 (0.0) -0.1 (0.1) 0.0 (0.0) 
G8Y11 England Exp —-0.1 (0.5) —1.1 (0.5) —0.4 (0.3) 
G8Y15 Australia Alignment 33.8 (20.8) 101.3 (44.7) 24.9 (19.5) 
G8Y15 Botswana Prepared 9.7 (5.0) 15.6 (4.9) 5.5 (5.4) 
G8Y15 Ireland Mathprep 3.0 (2.7) 4.7 (2.3) 3.0 (2.2) 
G8Y15 Korea Alignment 17.6 (28.9) 85.4 (38.1) 11.0 (24.4) 
G8Y15 Kuwait Prepared 15.6 (10.0) 9.7 (4.7) 9.4 (10.8) 
G8Y15 Lebanon Mathprep 4.7 (2.7) 4.2 (2.0) 2.0 (2.4) 
G8Y15 Lithuania Alignment 4.5 (31.1) 96.1 (46) 62.1 (47.8) 
G8Y15 Lithuania Prepared 4.1 (4.8) 10 (4.8) 10.8 (8.4) 
G8Y15 Malta Mathtime 0.0 (0.0) 0.1 (0.0) 0.1 (0.1) 
G8Y15 Norway Alignment | —14.5 (13.5) | —22.6 (10.3) | —18.4 (16.4) 
G8Y15 Qatar Exp —0.4 (0.6) 1.1 (0.5) —0.1 (0.6) 
G8Y15 Singapore Prepared —3.9 (3.8) -8.7 (4.1) —-1.2 (3.3) 
G8Y15 South Africa Prepared 21.6 (11.9) 31.4 (4.2) 16.8 (8.6) 
G8Y15 Sweden Prepared 3.6 (4.2) 10.3 (4.3) 2.4 (5.1) 
G8Y15 Thailand Exp —0.3 (0.4) —0.6 (0.3) —0.5 (0.3) 
G8Y15 Argentina (Buenos Aires) Alignment | —99.1 (64.0) | —120.8 (38.8) | —12.0 (67.4) 
G8Y15 Argentina (Buenos Aires) Prepared 13.6 (9.4) 15.4 (7.0) 12.0 (8.8) 


Notes Standard errors in parentheses. Grade/year indicates grade level (where G4 and G8 are grade four 
and eight, respectively) and year (where e.g., YO7 is the year 2007), resulting in a composite abbreviation 
such as G4Y07 (which indicates Grade 4, 2007 cycle of TIMSS data). SLM = single level model, MLM = 
multilevel model, CLM = classroom-level model, Alignment = proportion of mathematics topics covered 
compared with national expectations, Mathprep = index (1-5) of teacher education to teach mathematics, 
Mathtime = mean number of minutes spent teaching mathematics per week, Prepared = index (1-4) of 
self-efficacy to teach mathematics, Exp = experience teaching in years 
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Table B.9 Countries where the relationship of a given indicator of teacher quality with student 
achievement was statistically significant for only the single level model 


Grade/year | Country Predictor SLM MLM CLM 

G4Y03 New Zealand Prepared 21.9 (8.6) 0.4 (7.1) 17.1 (8.8) 
G4Y07 Denmark Alignment 32.6 (16.2) 16.0 (13.8) 35.8 (20.3) 
G4Y07 Hungary Alignment | —113.5 (57.9) | -125.3 (68.4) | —101.2 (52.3) 


G4Y07 


Lithuania 


Prepared 


8.5 (3.7) 


2543.9) 


5.9 (5.1) 


G4Y07 New Zealand Mathtime 0.1 (0.0) 0.0 (0.0) 0.0 (0.0) 
G4Y07 Slovenia Prepared —5.2 (2.6) —2.7 (3.0) -4.5 (3.1) 
G4Y07 United Arab Emirates (Dubai) Mathtime 0.3 (0.1) 0.2 (0.1) 0.1 (0.2) 
G4Y07 United States Exp 0.4 (0.2) 0.4 (0.2) 0.3 (0.2) 
G4Y11 Botswana Prepared 16.2 (8.1) 15.4 (8.1) 11 (8.7) 
G4yY11 Georgia Alignment 54.1 (27.2) 40.1 (24.7) 72.1 (44.4) 
G4yY11 Georgia Prepared 11.7 (5.0) 5.3 (5.0) 9.1 (7.2) 
G4yY11 Hungary Alignment —67.8 (25.7) —36.3 (21.8) —53.7 (29.8) 
G4Y11 Iran, Islamic Republic of Prepared 11.8 (6.0) 1.8 (4.6) 5.4 (5.6) 
G4yY11 Oman Prepared -17.2 (6.1) —4.7 (4.3) —13.0 (7.4) 
G4yY11 Serbia Mathprep 7.7 3.1) 3.5 (3.5) —2.0 (4.8) 
G4yY11 Spain Exp 0.6 (0.2) 0.2 (0.2) 0.3 (0.2) 
G4yY15 Bahrain Prepared 13.9 (4.3) 2.4 (5.9) 9.3 (5.4) 
G4Y15 France Exp 0.9 (0.3) 0.6 (0.3) 0.1 (0.5) 
G4Y15 Northern Ireland Exp 0.7 (0.3) 0.5 (0.3) 0.3 (0.3) 
G4yY 15 Sweden Alignment 32.6 (11.4) 6.2 (12.2) 23.5 (12.9) 
G4yY 15 United Arab Emirates Exp 1.0 (0.5) 0.1 (0.4) 0.7 (0.5) 
G4yY15 United Arab Emirates Mathtime 0.1 (0.1) 0.0 (0.0) 0.1 (0.1) 
G4yY 15 United Arab Emirates (Dubai) Prepared 27.3 (5.4) 5.1 (13.2) 12.1 (7.8) 
G4yY 15 United States Prepared 12.4 (4.9) 7.7 (4.0) 4.3 (4.2) 
G8Y03 Canada (Ontario) Exp 0.6 (0.3) 0.4 (0.2) 0.3 (0.4) 
G8Y03 Indonesia Prepared 41.8 (20) 0.9 (3.7) 37.6 (19.4) 
G8Y03 Sweden Prepared 19.2 (8.6) 5.7 (3.1) 11.0 (10.2) 
G8Y07 Armenia Exp —0.6 (0.3) —0.3 (0.3) -0.5 (0.4) 
G8Y07 Bahrain Mathtime 0.1 (0.0) 0.0 (0.0) 0.1 (0.0) 
G8Y07 Canada (British Columbia) Mathprep 5.6 (2.6) 5.7 (3.6) 6.4 (3.4) 
G8Y07 Canada (British Columbia) Mathtime -0.2 (0.1) —0.1 (0.1) -0.1 (0.1) 
G8Y07 Czech Republic Mathprep 5.0 (1.5) 2.2 (2.3) 2.5 (1.5) 
G8Y07 England Exp —-0.9 (0.4) —0.7 (0.5) —-0.3 (0.3) 
G8Y07 England Mathprep -8.7 (3.7) —7.4 (7.5) -6.5 (4.8) 
G8Y07 England Mathtime —0.3 (0.1) 0.0 (0.2) —-0.1 (0.2) 
G8Y07 Iran, Islamic Republic of Prepared 21.6 (9.5) —-1.5 (2.7) 10.9 (7.5) 
G8Y07 Lebanon Alignment 82.8 (33.7) 39.4 (24.8) 35.9 (42.4) 
G8Y07 Lithuania Prepared 12.1 (3.5) —0.1 (3.4) 5.4 (4.4) 
G8Y07 Malaysia Prepared 20.6 (8.0) —3.0 (5.5) 2.7 (7.2) 
G8Y07 Malta Exp 1.0 (0.2) —0.2 (0.2) 0.9 (0.5) 
G8Y07 Malta Mathprep —7.0 (1.2) 0.7 (1.5) -4.8 (3.4) 
G8Y07 Malta Mathtime —0.2 (0.1) —0.1 (0.1) 0.1 (0.2) 
G8Y07 Malta Prepared 18.3 (2.4) 0.7 (1.5) 3.0 (10.4) 
G8Y07 Qatar Mathprep —2.8 (1.3) 0.7 (1.7) —4.2 (3.5) 
G8Y07 Romania Exp 0.7 (0.3) 0.2 (0.4) 0.6 (0.4) 
G8Y07 Serbia Prepared 14.8 (6.8) 0.9 (3.5) 11.2 (7.3) 
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Table B.9 (continued) 
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Grade/year | Country Predictor SLM MLM CLM 

G8Y07 Sweden Prepared 9.7 (4.1) 1.7 (1.8) 12.8 (6.8) 
G8Y07 Syria, Arab Republic of Prepared —23.0 (7.6) 6.0 (5.3) -18.9 (9.9) 
G8Y07 Thailand Prepared 20.1 (8.3) —1.8 (3.4) 2.4 (7.6) 
G8Y07 Ukraine Prepared 16.0 (8.1) 1.4 (4.8) 12.0 (8.2) 
G8Y07 United Arab Emirates (Dubai) Alignment | —132.8 (58.9) 24.8 (105.8) -72.7 (66) 
G8Y07 United States (Massachusetts) Mathtime —0.2 (0.1) 0.0 (0.1) -0.1 (0.1) 
G8Y11 Australia Mathtime -0.2 (0.1) —0.1 (0.1) -0.1 (0.1) 
G8Y11 Bahrain Mathprep 14.4 (3.4) 1.8 (6.3) 0.6 (3.8) 
G8Y11 Bahrain Mathtime 0.1 (0.0) 0.0 (0.1) 0.1 (0.1) 
G8Y11 Chinese Taipei Mathtime 0.2 (0.1) 0.1 (0.1) 0.0 (0.1) 
G8Y11 England Mathtime —0.4 (0.2) —0.2 (0.1) —-0.1 (0.1) 
G8Y11 Hong Kong, SAR Prepared —10.6 (4.8) —9.8 (7.6) —2.6 (4.1) 
G8Y11 Lebanon Alignment 87.3 (32.2) 48.2 (26.9) 27.6 (32.3) 
G8Y11 New Zealand Alignment 88.3 (28.1) —9.8 (40.1) 32.0 (18.9) 
G8Y11 New Zealand Prepared 15.6 (6.7) 5 (11.8) 2.6 (7.0) 
G8Y11 Saudi Arabia Alignment 68.3 (32.1) 48.7 (27.3) 56.7 (31.7) 
G8Y11 United Arab Emirates (Dubai) Mathprep 10.3 (3.7) 1.8 (6.7) 6.8 (3.7) 
G8Y11 United Arab Emirates (Dubai) Mathtime —-0.1 (0.1) 0.0 (0.1) —-0.1 (0.1) 
G8Y11 United States Exp 0.5 (0.3) —0.3 (0.5) 0.3 (0.3) 
G8Y15 Canada (Ontario) Exp -0.7 (0.3) —0.3 (0.5) —0.1 (0.4) 
G8Y 15 Canada (Ontario) Mathtime —0.1 (0.0) 0.0 (0.0) 0.0 (0.0) 
G8Y15 Canada (Quebec) Mathtime —0.1 (0.0) —0.2 (0.2) 0.0 (0.1) 
G8Y15 Chinese Taipei Mathprep 6.3 (2.0) 4.3 (4.5) 0.2 (4.8) 
G8Y15 Hong Kong, SAR Mathprep —8.9 (3.3) —1 4.1) —6.3 (3.6) 
G8Y15 Japan Prepared —5.5 (2.4) —3.4 (3.0) -1.5 (2.5) 
G8Y 15 Malta Mathprep —5.6 (1.4) —2.5 (1.7) 5.0 (4.8) 
G8Y15 New Zealand Exp 0.6 (0.3) 0.3 (0.4) 0.1 (0.3) 
G8Y15 New Zealand Prepared 9.7 (4.7) 9.8 (7.0) 8.3 (4.4) 
G8Y15 Qatar Alignment —59.4 (25.6) 10.5 (31.1) —34.6 (24.2) 
G8Y 15 Qatar Mathtime —0.1 (0.0) 0.0 (0.1) —-0.1 (0.0) 
G8Y15 United Arab Emirates (Abu Prepared 26.8 (11.4) 6.4 (10.1) 6.8 (11) 

Dhabi) 

G8Y15 United Arab Emirates (Dubai) Mathtime —0.2 (0.0) 0.0 (0.1) -0.1 (0.1) 
G8Y15 United Arab Emirates (Dubai) Prepared 16.0 (6.0) —4.3 (6.0) 5.3 (5.4) 


Notes Standard errors in parentheses. Grade/year indicates grade level (where G4 and G8 are grade four and eight, 
respectively) and year (where e.g., YO7 is the year 2007), resulting in a composite abbreviation such as G4Y07 
(which indicates Grade 4, 2007 cycle of TIMSS data). SLM = single level model, MLM = multilevel model, CLM = 
classroom-level model, Alignment = proportion of mathematics topics covered compared with national expectations, 
Mathprep = index (1-5) of teacher education to teach mathematics, Mathtime = mean number of minutes spent 
teaching mathematics per week, Prepared = index (1-4) of self-efficacy to teach mathematics, Exp = experience 
teaching in years 
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Table B.10 Countries where the relationship of a given indicator of teacher quality with student 
achievement was statistically significant for only the classroom means model 


Grade/year | Country Predictor SLM MLM CLM 
G4Y07 Canada (British Columbia) Exp 0.5 (0.3) 0.4 (0.2) 0.6 (0.3) 
G4Y07 Czech Republic Mathtime —0.2 (0.2) —0.1 (0.2) —0.4 (0.2) 
G4Y07 Lithuania Alignment | 33.4 (25.7) 10.3 (19.8) | 61.7 (29.6) 
G4Y07 New Zealand Mathprep —-6.6 (4.1) —4.0 (4.2) —8.3 (3.6) 
G4Y07 United States (Minnesota) Prepared —12.1 (18.3) 4.8 (16.7) | -30.3 (15.3) 
G4Y11 Denmark Prepared —12.0 (14.8) | -8.0 (5.4) | -18.4 (9.3) 
G4Y11 England Prepared 15.4 (9.6) 19.7 (11.8) | 27.6 (10.9) 
G4Y11 Hungary Mathprep —2.4 (9.1) | -10.4 (17.2) | —26.2 (12.0) 
G4yY15 Bahrain Mathtime 0.0 (0.0) 0.0 (0.0) 0.1 (0.0) 
G4yY15 Canada (Ontario) Mathprep 3.6 (2.9) 0.2 (2.8) 9.7 (3.8) 
G4yY15 Denmark Mathtime 0.1 (0.1) 0.1 (0.1) 0.1 (0.1) 
G4yY15 Finland Exp 0.2 (0.1) 0.2 (0.2) 0.6 (0.2) 
G4yY15 Lithuania Alignment 18.9 (16.4) 9.5 (19.3) | 42.5 (19) 
G4Y15 Slovak Republic Mathprep 6.6 (3.6) 1.4 (3.6) 10.2 (3.9) 
G4yY15 Sweden Prepared 5.0 (2.7) 5.4 (2.9) 7.3 (3.4) 
G4yY15 United States Alignment | 28.8 (21.3) 25.8 (20.6) 38 (16.8) 
G8Y07 Bulgaria Mathtime —0.1 (0.3) -0.1 (0.1) 0.2 (0.1) 
G8Y07 Hong Kong, SAR Mathtime —-0.1 (0.1) -0.2 (0.1) —-0.1 (0.1) 
G8Y07 Indonesia Mathprep 7.6 (4.5) 7.1 (3.8) 9.5 (4.2) 
G8Y07 Lebanon Exp 0.8 (0.5) 0.5 (0.3) 1.3 (0.4) 
G8Y07 Scotland Exp 0.2 (0.3) 0.3 (0.4) 0.6 (0.2) 
G8Y07 United Arab Emirates Mathprep 19.1 (12.1) 16.9 (15.3) | 28.8 (12.5) 
(Dubai) 
G8Y07 United Arab Emirates Prepared —-19.8 (13.6) 16.9 (15.3) | —43.4 (17) 
(Dubai) 
G8Y07 United States (Minnesota) Mathtime 0.1 (0.1) 0.1 (0.1) 0.3 (0.1) 
G8Y11 Bahrain Alignment 15.1 (15.1) —13 (36.2) —47 (23.3) 
G8Y11 Canada (Alberta) Mathtime 0.0 (0.0) 0.1 (0.1) 0.1 (0.0) 
G8Y11 Hungary Exp 0.6 (0.3) 0.4 (0.3) 0.8 (0.4) 
G8Y11 Morocco Exp 0.0 (0.3) 0.0 (0.2) —0.9 (0.3) 
G8Y11 Morocco Mathprep 2.0 13.0) -1@.7) —7.0 (2.9) 
G8Y11 Sweden Mathprep 0.7 (1.7) 0.7 (1.8) 3.7 (1.7) 
G8Y11 United Arab Emirates (Abu | Alignment 11.4 (26.6) 22.1 (31.0) | 48.9 (23.0) 
Dhabi) 
G8Y15 Bahrain Alignment | —1.1 (7.2) 2.1 (12.9) 18.9 (8.6) 
G8Y15 Bahrain Prepared 5.5 (4.7) -0.7 (5.2) 8.6 (3.8) 
G8Y15 England Mathtime —-0.1 (0.1) 0.1 (0.3) 0.2 (0.1) 
G8Y15 Israel Alignment | 57.6 (37.1) 57.9 (36) 95.9 (29.2) 
G8Y15 Kuwait Mathprep 7.1 (6.5) 3.4 (2.8) 11.9 (4.3) 
G8Y15 Slovenia Mathtime —0.1 (0.0) —-0.1 (0.0) —0.1 (0.0) 
G8Y15 Turkey Mathtime 0.0 (0.1) 0.0 (0.1) 0.1 (0.0) 
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Table B.10 (continued) 


Grade/year | Country Predictor SLM MLM CLM 

G8Y15 United Arab Emirates (Abu | Mathprep 11.1 (9.0) —3.4 (8.0) 15.4 (6.1) 
Dhabi) 

G8Y15 United Arab Emirates Alignment 1.6 (15.3) | -11.7 (24.4) 41.6 (17.7) 
(Dubai) 


Notes Standard errors in parentheses. Grade/year indicates grade level (where G4 and G8 are grade four 
and eight, respectively) and year (where e.g., YO7 is the year 2007), resulting in a composite abbreviation 
such as G4Y07 (which indicates Grade 4, 2007 cycle of TIMSS data). SLM = single level model, MLM = 
multilevel model, CLM = classroom-level model, Alignment = proportion of mathematics topics covered 
compared with national expectations, Mathprep = index (1-5) of teacher education to teach mathematics, 
Mathtime = mean number of minutes spent teaching mathematics per week, Prepared = index (1-4) of 
self-efficacy to teach mathematics, Exp = experience teaching in years 
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Table B.11 Countries where the relationship of teacher experience (Exp) with student 
achievement was not statistically significant in all three models 


Grade/year | Country Predictor | SLM MLM CLM 

G4Y03 Cyprus Exp —-0.1 (0.4) | -0.1 (0.3) | -0.1 (0.4) 
G4Y03 Hong Kong Exp 0.4 (0.2) 0.7 (0.4) 0.5 (0.3) 
G4Y03 Japan Exp 0.3 (0.2) 0.2 (0.1) 0.3 (0.2) 
G4Y03 Netherlands Exp -0.1 (2.2) 
G4Y03 New Zealand Exp 0.5 (0.3) 0.4 (0.4) 0.4 (0.3) 
G4Y03 Singapore Exp 0.1 (0.3) 0.1 (0.2) 0.1 (0.2) 
G4Y03 Slovenia Exp —0.3 (0.3) | -0.1 (0.2) | -0.3 (0.3) 
G4Y03 Yemen Exp 0.2 (1.1) | -0.1 (0.4) 0.1 (1.0) 
G4Y03 Canada (Ontario) Exp 0.5 (0.3) 0.3 (0.3) 0.2 (0.3) 
G4Y03 Canada (Quebec) Exp 0.0 (0.2) 0.0 (0.2) 0.1 (0.3) 
G4Y07 Australia Exp 0.1 (0.3) 0.2 (0.2) 0.4 (0.4) 
G4Y07 Austria Exp 0.3 (0.2) 0.1 (0.2) 0.1 (0.3) 
G4Y07 Armenia Exp 0.1 (0.5) 0.3 (0.2) | —0.3 (0.8) 
G4Y07 Colombia Exp —0.2 (0.6) | -0.6 (0.5) | —0.7 (0.6) 
G4Y07 Czech Republic Exp 0.1 (0.2) 0.2 (0.2) 0.1 (0.2) 
G4Y07 Denmark Exp —0.3 (0.3) | —0.2 (0.2) | —0.2 (0.3) 
G4Y07 El Salvador Exp 0.9 (0.6) 0.9 (0.5) 0.3 (0.7) 
G4Y07 Georgia Exp —0.7 (0.4) —0.4 (0.4) —0.8 (0.6) 
G4Y07 Germany Exp —-0.2 (0.2) | -0.2 (0.1) | —0.1 (0.2) 
G4Y07 Hong Kong Exp 0.4 (0.3) 0.4 (0.4) | —0.2 (0.3) 
G4Y07 Hungary Exp —0.1 (0.5) 0.3 (0.6) | —0.5 (0.4) 
G4Y07 Japan Exp 0.2 (0.2) 0.2 (0.1) 0.3 (0.2) 
G4Y07 Mongolia Exp 0.4 (0.5) 0.1 (0.3) 0.3 (0.5) 
G4Y07 Morocco Exp 0.4 (0.7) 0.5 (0.4) | —0.4 (0.9) 
G4Y07 Netherlands Exp —0.2 (0.9) | —0.1 (0.4) 0.8 (0.6) 
G4Y07 New Zealand Exp 0.1 (0.2) 0.2 (0.2) 0.4 (0.3) 
G4Y07 Singapore Exp —0.1 (0.4) | —0.2 (0.5) | —0.3 (0.3) 
G4Y07 Slovenia Exp 0.0 (0.2) 0.1 (0.2) | —0.1 (0.2) 
G4Y07 Sweden Exp 0.1 (0.2) | —0.1 (0.2) 0.0 (0.2) 
G4Y07 Yemen Exp -1.2 (1.1) |-1.3 0.9) | -2.4 1.3) 
G4Y07 Scotland Exp 0.0 (0.2) | —0.1 (0.2) 0.1 (0.3) 
G4Y07 United Arab Emirates (Dubai) Exp —1.8 (1.0) 0.3 (0.6) | 1.6 (1.3) 
G4Y07 Canada (Ontario) Exp 0.0 (0.3) 0.2 (0.4) | -0.2 (0.4) 
G4Y07 Canada (Quebec) Exp 0.1 (0.3) 0.1 (0.3) 0.2 (0.4) 
G4Y07 Canada (Alberta) Exp —0.1 (0.3) 0.0 (0.2) 0.0 (0.2) 
G4Y07 United States (Massachusetts) Exp —0.4 (0.4) | —0.1 (0.3) 0.0 (0.4) 
G4Y07 United States (Minnesota) Exp 0.3 (0.4) 0.9 (0.7) 0.2 (0.4) 
G4Y11 Australia Exp —-0.1 (0.3) | -0.3 (0.3) 0.0 (0.4) 
G4yY11 Bahrain Exp 0.0 (0.5) | —0.2 (0.4) | —0.3 (0.5) 
G4Y11 Chile Exp —0.2 (0.3) 0.0 (0.2) | —0.4 (0.4) 
G4Y11 Chinese Taipei Exp 0.2 (0.3) 0.1 (0.2) 0.1 (0.3) 
G4yY11 Croatia Exp 0.3 (0.2) 0.2 (0.2) 0.1 (0.2) 
G4Y11 Czech Republic Exp —0.1 (0.2) 0.1 (0.2) 0.2 (0.3) 
G4Y11 Denmark Exp 0.1 (0.2) 0.1 (0.1) 0.1 (0.2) 
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Table B.11 (continued) 
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Grade/year | Country Predictor | SLM MLM CLM 

G4Y11 Finland 0.3 (0.2) 0.2 (0.2) 0.5 (0.3) 
G4Y11 Georgia —0.2 (0.3) | -0.2 (0.3) | -0.7 (0.4) 
G4Y11 Germany 0.0 (0.1) —0.1 (0.1) 0.0 (0.2) 
G4Y11 Honduras 0.3 (0.6) 0.7 (0.5) —0.1 (0.7) 
G4Y11 Hungary 0.2 (0.3) 0.4 (0.3) 0.5 (0.4) 
G4Y11 Treland 0.5 (0.3) 0.2 (0.2) 0.6 (0.4) 
G4Y11 Italy 0.4 (0.3) 0.5 (0.3) 0.4 (0.3) 
G4Y11 Japan 0.1 (0.1) 0.1 (0.1) 0.2 (0.2) 
G4Y11 Korea 0.2 (0.1) 0.2 (0.1) 0.1 (0.1) 
G4Y11 Kuwait 0.6 (0.8) 1.3 (0.7) 0.9 (0.8) 
G4Y11 Lithuania —-0.5 (0.3) | -0.2 (0.2) | —0.3 (0.3) 
G4Y11 Oman 0.5 (0.6) 0.3 (0.3) 0.3 (0.7) 
G4yY11 Netherlands 0.0 (0.2) 0.1 (0.2) 0.3 (0.2) 
G4Y11 New Zealand —0.2 (0.3) 0.1 (0.3) | —0.1 (0.4) 
G4Y11 Norway 0.0 (0.2) 0.0 (0.2) 0.3 (0.2) 
G4Y11 Romania 0.4 (0.4) 0.1 (0.4) 0.2 (0.5) 
G4Y11 Saudi Arabia 0.4 (0.6) 0.6 (0.4) | —0.3 (0.6) 
G4Y11 Serbia -0.5 (0.3) | -0.3 (0.3) | 1.2 (0.7) 
G4Y11 Slovak Republic 0.0 (0.3) | —0.2 (0.2) 0.1 (0.4) 
G4yY11 Sweden 0.1 (0.2) 0.2 (0.2) 0.0 (0.2) 
G4Y11 Thailand 0.0 (0.4) 0.1 (0.2) | —0.2 (0.5) 
G4Y11 United Arab Emirates —0.1 (0.6) —0.3 (0.3) 0.1 (0.6) 
G4Y11 United States 0.2 (0.2) 0.1 (0.2) 0.0 (0.2) 
G4Y11 Yemen 0.4 (1.0) 0.5 (0.5) 0.5 (1.0) 
G4yY11 England 0.6 (0.3) 0.0 (0.3) 0.4 (0.4) 
G4Y11 0.1 (0.3) 
G4Y11 United Arab Emirates (Dubai) 0.0 (0.8) —0.5 (0.4) 0.0 (0.9) 
G4Y11 United Arab Emirates (Abu Dhabi) —-0.1 (0.8) | -0.8 (0.7) | —0.2 (0.8) 
G4Y11 Canada (Ontario) —-0.1 (0.3) | -0.3 (0.3) | -0.5 (0.4) 
G4Y11 Canada (Quebec) 0.0 (0.2) 0.0 (0.2) 0.0 (0.3) 
G4Y11 Canada (Alberta) 0.1 (0.3) 0.3 (0.2) | —0.1 (0.3) 
G4Y15 Bulgaria 0.1 (0.4) 0.1 (0.5) 0.2 (0.4) 
G4Y15 Canada —0.1 (0.2) 0.0 (0.2) | -0.1 (0.3) 
G4Y15 Chile 0.0 (0.6) 0.0 (0.3) 0.2 (0.4) 
G4Y15 0.1 (0.3) 
G4Y15 Cyprus 0.6 (0.3) | —0.1 (0.3) 0.4 (0.3) 
G4Y15 Czech Republic 0.0 (0.2) 0.0 (0.2) | —0.2 (0.2) 
G4Y15 Denmark —0.1 (0.3) 0.0 (0.2) | —0.2 (0.4) 
G4Y15 Georgia —0.2 (0.4) 0.1 (0.3) 0.2 (0.5) 
G4Y15 Hong Kong —0.2 (0.3) 0.0 (0.2) 0.2 (0.3) 
G4Y15 Hungary —0.3 (0.3) 0.0 (0.3) | —0.1 (0.4) 
G4Y15 Indonesia 0.1 (0.5) 0.2 (0.4) 0.3 (0.6) 
G4Y15 Tran 0.4 (0.6) 0.4 (0.3) 0.0 (0.5) 
G4Y15 0.3 (0.4) 
G4Y15 Italy Exp 0.1 (0.3) 0.3 (0.3) 0.1 (0.3) 
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Grade/year | Country Predictor | SLM MLM CLM 
G4Y15 Japan 0.0 (0.1) | -0.1 (0.1) 0.2 (0.1) 
G4Y15 Kazakhstan —0.5 (0.5) | —0.6 (0.4) | -0.5 (0.5) 
G4Y15 Kuwait 0.2 (0.8) 0.0 (0.6) 0.0 (0.7) 
G4Y15 Lithuania 0.0 (0.3) 0.1 (0.2) 0.1 (0.3) 
G4Y15 Oman 0.3 (0.8) 0.0 (0.7) 0.4 (0.9) 
G4Y15 New Zealand 0.0 (0.3) 0.0 (0.2) 0.1 (0.3) 
G4Y15 Norway 0.4 (0.2) 0.1 (0.2) 0.3 (0.3) 
G4Y15 Poland 0.4 (0.2) 0.2 (0.2) 0.4 (0.3) 
G4Y15 Portugal 0.0 (0.3) 0.1 (0.2) | -0.2 (0.3) 
G4Y15 Qatar 0.1 (0.5) 0.2 (0.6) 0.4 (0.6) 
G4Y15 Serbia 0.0 (0.3) | —0.1 (0.3) | —0.1 (0.5) 
G4Y15 Slovak Republic 0.1 (0.2) 0.1 (0.2) | —0.1 (0.3) 
G4Y15 Slovenia 0.2 (0.2) 0.2 (0.2) 0.2 (0.2) 
G4Y15 Sweden 0.0 (0.2) | —-0.1 (0.1) 0.2 (0.2) 
G4Y15 England 0.3 (0.4) 0.1 (0.4) | —-0.1 (0.4) 
G4Y15 United Arab Emirates (Dubai) 0.3 (0.5) 0.1 (0.8) 0.0 (0.5) 
G4Y15 Canada (Ontario) —0.4 (0.3) | -0.2 (0.3) 0.0 (0.3) 
G4Y15 Canada (Quebec) —0.4 (0.4) —0.1 (0.3) —0.4 (0.4) 
G8Y03 Australia -0.7 (0.5) | —0.3 (0.3) 0.0 (0.5) 
G8Y03 Bulgaria —0.7 (0.6) | —-0.4 (0.6) | —0.8 (0.6) 
G8Y03 Chile 0.1 (0.4) 0.0 (0.2) 0.2 (0.3) 
G8Y03 Chinese Taipei 0.0 (0.4) 0.4 (0.5) | —0.3 (0.3) 
G8Y03 Cyprus 0.3 (0.2) 0.4 (0.2) 0.3 (0.3) 
G8Y03 Estonia —0.4 (0.2) | -0.3 (0.2) | -0.2 (0.2) 
G8Y03 Hong Kong 0.7 (0.4) 0.5 (0.4) 0.3 (0.4) 
G8Y03 0.0 (0.3) 
G8Y03 Indonesia 0.9 (1.1) 0.2 (0.6) 0.9 (0.9) 
G8Y03 Japan 0.1 (0.4) 0.1 (0.2) 0.2 (0.3) 
G8Y03 Lithuania —0.1 (0.3) 0.2 (0.3) 0.3 (0.3) 
G8Y03 Malaysia 0.0 (0.6) 0.2 (0.5) | —0.5 (0.4) 
G8Y03 Moldova 0.5 (0.5) 0.5 (0.4) 0.5 (0.6) 
G8Y03 New Zealand —0.1 (0.4) 0.2 (0.5) | —0.6 (0.4) 
G8Y03 Norway -0.1 (0.3) 0.0 (0.2) | —0.1 (0.3) 
G8Y03 Singapore 0.2 (0.3) 0.0 (0.5) 0.0 (0.2) 
GRY 03 0.104 
G8Y03 Sweden —0.1 (0.2) 0.3 (0.3) 0.2 (0.3) 
G8Y03 Macedonia 0.5 (0.4) 0.5 (0.3) 0.6 (0.4) 
G8Y03 Serbia 0.2 (0.2) 0.3 (0.2) 0.2 (0.2) 
G8Y03 Canada (Quebec) 0.0 (0.3) 0.2 (0.2) | —0.2 (0.2) 
G8Y07 Algeria —0.2 (0.3) | -0.3 (0.2) | -0.2 (0.3) 
G8Y07 Australia 0.7 (0.5) 0.5 (0.4) 0.2 (0.4) 
G8Y07 Bahrain 0.4 (0.3) 0.0 (0.4) 0.4 (0.4) 
G8Y07 Bosnia and Herzegovina 0.2 (0.2) 0.2 (0.2) | —0.2 (0.2) 
G8Y07 0.0 (0.5) 
G8Y07 Chinese Taipei Exp —0.1 (0.4) —0.9 (0.9) —0.5 (0.4) 
G8Y07 Colombia Exp 0.2 (0.3) 0.3 (0.3) 0.0 (0.4) 
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Grade/year | Country Predictor | SLM MLM CLM 

G8Y07 Czech Republic 0.2 (0.2) 0.2 (0.3) 0.2 (0.2) 
G8Y07 Georgia —0.4 (0.4) | -0.2 (0.3) | -0.5 (0.5) 
G8Y07 Hong Kong 0.3 (0.6) | —0.7 (0.6) 0.6 (0.5) 
G8Y07 Japan —-0.2 (0.3) | -0.2 (0.1) | -0.1 (0.3) 
G8Y07 Jordan 0.7 (0.6) 0.6 (0.4) 0.5 (0.6) 
G8Y07 Korea 0.1 (0.3) 0.0 (0.5) | —0.1 (0.3) 
G8Y07 Kuwait 0.1 (0.5) 0.0 (0.5) | —0.3 (0.6) 
G8Y07 Lithuania —0.2 (0.3) | —0.1 (0.3) 0.2 (0.3) 
G8Y07 Malaysia 0.3 (0.5) 1.2 (1.3) | -0.3 (0.4) 
G8Y07 Oman 0.4 (0.9) 1.0 (0.7) 0.7 (1.0) 
G8Y07 Norway 0.1 (0.2) 0.0 (0.2) 0.2 (0.2) 
G8Y07 Saudi Arabia —0.2 (0.5) 0.2 (0.4) | —0.5 (0.5) 
G8Y07 Singapore 0.0 (0.3) | —0.3 (0.6) 0.1 (0.3) 
G8Y07 Slovenia 0.2 (0.2) 0.4 (0.2) 0.2 (0.2) 
G8Y07 Sweden 0.0 (0.2) | —0.2 (0.2) | —0.2 (0.2) 
G8Y07 0.8 (0.4) 0.7 (0.4) 0.7 (0.5) 
G8Y07 —-0.5 (0.6) | -0.2 (0.4) | —0.4 (0.5) 
G8Y07 —0.4 (0.3) | -0.1 (0.4) | -0.4 (0.3) 
G8Y07 0.1 (0.3) 0.3 (0.3) 0.1 (0.2) 
G8Y07 0.2 (0.3) 0.1 (0.1) 0.0 (0.2) 
G8Y07 pain (Basque) 0.0 (0.3) | —0.4 (0.3) 0.2 (0.3) 
G8Y07 United Arab Emirates (Dubai) —0.2 (0.8) | —0.3 (0.9) 0.7 (1.0) 
G8Y07 Canada (British Columbia) 0.5 (0.4) 0.7 (0.6) 0.5 (0.4) 
G8Y07 United States (Massachusetts) 0.9 (0.5) | —0.5 (0.7) 0.8 (0.4) 
G8Y07 United States (Minnesota) —0.2 (0.4) 0.2 (1.0) 0.5 (0.4) 
G8Y11 1.1 (0.5) 
G8Y11 Chile —0.1 (0.3) 0.1 (0.2) 0.2 (0.2) 
G8Y11 Chinese Taipei 0.5 (0.3) 1.0 (0.6) 0.3 (0.3) 
G8Y11 Georgia —-0.1 (0.3) | -0.2 (0.3) 0.1 (0.5) 
G8Y11 Honduras 0.2 (0.4) 0.3 (0.2) | —0.1 (0.5) 
G8Y11 Italy —-0.2 (0.2) | —-0.1 (0.2) | —-0.4 (0.2) 
G8Y11 Japan 0.0 (0.3) 0.3 (0.4) 0.1 (0.3) 
G8Y11 Jordan 0.5 (0.4) 0.5 (0.3) 0.5 (0.4) 
G8Y11 Lebanon 0.5 (0.4) 0.2 (0.2) 0.5 (0.3) 
G8Y11 0.0 (0.4) 
G8Y11 Malaysia 0.5 (0.6) 0.7 (0.5) 0.4 (0.4) 
G8Y11 Norway 0.1 (0.1) 0.0 (0.1) 0.1 (0.1) 
G8Y11 Romania 0.0 (0.3) | —-0.1 (0.5) | —0.3 (0.3) 
G8Y11 Saudi Arabia 1.0 (0.6) 0.0 (0.8) 0.2 (0.8) 
G8Y11 Singapore 0.3 (0.4) | -0.7 (0.7) | —0.2 (0.3) 
G8Y11 Slovenia —0.1 (0.2) 0.1 (0.2) | —0.2 (0.2) 
G8Y11 South Africa —-0.5 (0.4) | -0.5 (0.7) | -0.5 (0.4) 
G8Y11 Sweden 0.0 (0.2) | -0.2 (0.2) | —-0.1 (0.3) 
G8Y11 0.0 (0.4) 
G8Y11 Macedonia Exp 0.0 (0.5) 0.3 (0.6) 0.1 (0.5) 
G8Y11 United Arab Emirates (Dubai) Exp —0.6 (0.6) | —0.7 (0.6) | —0.6 (0.6) 
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Table B.11 (continued) 


Grade/year | Country Predictor | SLM MLM CLM 

G8Y11 United Arab Emirates (Abu Dhabi) —0.2 (0.8) 0.0 (0.9) —0.3 (0.6) 
G8Y11 Canada (Ontario) 0.0 (0.3) | —0.3 (0.5) 0.2 (0.3) 
G8Y11 Canada (Alberta) 0.0 (0.2) 0.1 (0.6) 0.0 (0.3) 
G8Y15 Australia 0.5 (0.3) | —0.9 (0.7) 0.1 (0.2) 
G8Y15 Bahrain 0.4 (0.2) =| -0.2 (0.2) | —0.3 (0.3) 
G8Y15 Botswana 0.0 (0.4) 0.5 (0.3) | —0.3 (0.4) 
G8Y15 Canada —-0.1 (0.3) | -0.3 (0.4) 0.2 (0.3) 
G8Y15 Chile 0.0 (0.3) 0.1 (0.2) | —0.2 (0.2) 
G8Y15 Georgia 0.3 (0.4) 0.2 (0.3) 1.0 (0.5) 
G8Y15 Hong Kong 0.5 (0.5) 1.2 (0.8) 0.6 (0.4) 
G8Y15 Hungary 0.0 (0.3) 0.2 (0.3) 0.3 (0.3) 
G8Y15 Treland 0.2 (0.2) 0.2 (0.3) 0.1 (0.2) 
G8Y15 Italy 0.4 (0.2) 0.2 (0.2) 0.2 (0.2) 
G8Y15 Japan 0.1 (0.2) 0.0 (0.2) 0.0 (0.2) 
G8Y15 Kazakhstan 0.7 (0.5) 0.1 (0.3) 0.4 (0.6) 
G8Y15 Jordan 0.4 (0.4) 0.3 (0.3) 0.1 (0.5) 
G8Y15 Kuwait —0.1 (0.7) 0.2 (0.3) 0.5 (0.7) 
G8Y15 Lebanon 0.7 (0.5) 0.3 (0.3) 0.5 (0.5) 
G8Y15 Lithuania —0.1 (0.3) 0.4 (0.3) 0.3 (0.4) 
G8Y15 Malaysia —0.2 (0.5) 1.1 (0.9) 0.0 (0.4) 
G8Y15 Oman 0.3 (0.5) 0.0 (0.2) 0.0 (0.4) 
G8Y15 Norway 0.1 (0.1) 0.2 (0.1) 0.1 (0.1) 
G8Y15 Singapore 0.2 (0.4) | -0.4 (0.5) | —0.2 (0.3) 
G8Y15 Slovenia 0.1 (0.2) 0.3 (0.2) 0.1 (0.2) 
G8Y15 South Africa —-0.5 (0.4) | -0.4 (0.3) | -0.4 (0.4) 
G8Y15 Sweden -0.2 (0.3) 
G8Y15 Egypt Exp 0.0 (0.5) | -0.2 (0.4) | —0.2 (0.5) 
G8Y15 United States Exp 0.2 (0.2) 0.0 (0.5) 0.1 (0.2) 
G8Y15 England Exp —0.6 (0.5) 0.5 (0.8) | —0.6 (0.3) 
G8Y15 United Arab Emirates (Abu Dhabi) | Exp 0.3 (0.9) | -0.9 (0.8) | —0.6 (0.7) 
G8Y15 Canada (Quebec) Exp —0.3 (0.4) | —0.6 (0.8) 0.1 (0.4) 
G8Y15 Argentina (Buenos Aires) Exp —0.3 (0.7) | —0.2 (0.5) | —0.4 (0.8) 


Notes Standard errors in parentheses. Grade/year indicates grade level (where G4 and G8 are grade four and 
eight, respectively) and year (where e.g., YO7 is the year 2007), resulting in a composite abbreviation such as 
G4Y07 (which indicates Grade 4, 2007 cycle of TIMSS data). SLM = single level model, MLM = multilevel 
model, CLM = classroom-level model, Exp = experience teaching in years 
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Table B.12 Countries where the relationship of formal preparation to teach mathematics 
(Mathprep) with student achievement was not statistically significant in all three models 


Grade/year | Country Predictor | SLM MLM CLM 
G4Y03 Chinese Taipei Mathprep | —1.0 (1.3) —1.0 (1.0) —0.6 (1.5) 
G4Y03 Cyprus Mathprep 4.7 (5.4) 10.1 (5.9) 6.5 (5.5) 
G4Y03 Hong Kong Mathprep 3.7 (2.6) —3.2 (4.0) 3.2 (2.6) 
G4Y03 Tran Mathprep 0.6 (2.1) 2.1 (2.0) -1.7 3.4) 
G4Y03 Japan Mathprep 2.9 (2.5) 2.7 (2.1) 3.4 (2.7) 
G4Y03 Netherlands Mathprep | —10.3 (7.0) | -10.7 (9.8) | —12.2 (13.9) 
G4Y03 New Zealand Mathprep 2.6 (5.5) 0.4 (7.1) 4.1 (5.4) 
G4Y03 Singapore Mathprep | —0.1 (2.6) 1.2 (2.1) 2.9 (1.9) 
G4Y03 Slovenia Mathprep | —2.2 (3.0) -0.8 (3.1) —-0.1 (4.4) 
G4Y03 Yemen Mathprep | —6.7 (5.3) —2.7 (1.7) —4.0 (5.5) 
G4Y03 Canada (Ontario) Mathprep | —3.8 (3.2) 0.6 (2.9) —4,3 (3.8) 
G4Y03 Canada (Quebec) Mathprep 2.6 (2.5) 2.7 (2.2) 0.9 (2.8) 
G4Y07 Algeria Mathprep 8.9 (5.8) 6.5 (3.1) 7.9 (6.3) 
G4Y07 Australia Mathprep 2.9 (4.6) 1.1 (5.6) 5.0 (9.9) 
G4Y07 Austria Mathprep 1.0 (3.2) 0.6 (2.2) —3.5 (4.3) 
G4Y07 Armenia Mathprep 10.9 (8.9) 4.4 (7.9) 19.3 (19.5) 
G4Y07 Chinese Taipei Mathprep | —0.9 (1.5) —0.2 (1.2) 0.1 1.5) 
G4Y07 Colombia Mathprep | —2.5 (4.6) 1.1 (2.7) —3.2 (5.8) 
G4Y07 Czech Republic Mathprep 2.0 (2.3) 2.5 (1.9) 2.6 (2.3) 
G4Y07 Denmark Mathprep 4.9 (2.2) 2.5 (2.0) 5.1 (2.6) 
G4Y07 EI Salvador Mathprep 8.3 (2.8) 8.8 (2.5) 3.7 (3.9) 
G4Y07 Georgia Mathprep | —0.6 (5.2) -7.0 (3.0) -0.1 (7.1) 
G4Y07 Germany Mathprep 11.8 (5.5) 13.1 (5.6) 16 (16.7) 
G4Y07 Hong Kong Mathprep 4.5 (3.2) 0.5 (4.7) 5.4 (2.8) 
G4Y07 Hungary Mathprep | —13.1 (19.8) | —18.5 (31.2) | —5.6 (13.7) 
G4Y07 Tran Mathprep 3.1 (1.7) 4.9 (1.4) 1.5 (2.1) 
G4Y07 Japan Mathprep 0.0 (2.2) —-0.2 (1.9) 0.2 (2.6) 
G4Y07 Lithuania Mathprep 3.8 (5.1) 2.5 (3.9) —-1.6 (4.7) 
G4Y07 Mongolia Mathprep | —0.5 (3.5) —2.1 (2.5) —0.2 (4.0) 
G4Y07 Netherlands Mathprep | —1.7 (3.5) —1.8 (2.2) —2.0 (3.6) 
G4Y07 New Zealand Mathprep | —6.6 (4.1) —4.0 (4.2) —8.3 (3.6) 
G4Y07 Singapore Mathprep | —2.9 (3.2) —3.2 (4.2) —0.7 (3.0) 
G4Y07 Slovenia Mathprep | —3.5 (2.7) —2.7 (3.0) —3.6 (3.0) 
G4Y07 Sweden Mathprep 0.6 (2.4) 1.5 (2.4) 1.6 (3.6) 
G4Y07 Tunisia Mathprep 8.2 3.1) 7.0 (2.9) 7.0 (5.0) 
G4Y07 United States Mathprep 1.4 (2.8) 3.4 (3.1) 4.5 (2.7) 
G4Y07 Yemen Mathprep | —0.3 (5.7) -0.3 (3.4) —4.6 (6.6) 
G4Y07 England Mathprep | —3.5 (3.5) -1.7 (3.3) —1.2 (4.2) 
G4Y07 Scotland Mathprep 3.1 (4.3) 1.6 (3.8) 6.2 (5.1) 
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Grade/year | Country Predictor | SLM MLM CLM 
G4Y07 United Arab Emirates Mathprep 13.3 (8.1) —0.9 (8.4) —1.6 (15.0) 
(Dubai) 
G4Y07 Canada (Ontario) Mathprep | —1.4 (3.8) 1.2 (2.9) 0.0 (3.4) 
G4Y07 Canada (Quebec) Mathprep 2.4 (5.3) 1.8 (4.6) 1.3 (5.0) 
G4Y07 Canada (Alberta) Mathprep | —0.1 (3.3) —3.1 (2.9) 2.0 (3.4) 
G4Y07 Canada (British Columbia) | Mathprep 1.9 (3.3) 0.5 (3.4) —0.2 (3.3) 
G4Y07 United States Mathprep | —5.5 (3.1) —4.1 (3.6) —1.0 (5.5) 
(Massachusetts) 
G4Y07 United States (Minnesota) Mathprep 13.6 (20) 4.8 (16.7) | —3.3 (16.6) 
G4Y11 Australia Mathprep 4.9 (7.7) -7.4 (7.8) —5.7 (6.7) 
G4Y11 Bahrain Mathprep | —3.9 (4.2) —-1.9 (3.0) —3.4 (4.6) 
G4yY11 Botswana Mathprep | —4.2 (3.5) —3.8 (2.7) —6.0 (4.1) 
G4Y11 Chile Mathprep | 9.3 (7.1) 6.2 (10.4) | 2.7 (7.6) 
G4Y11 Chinese Taipei Mathprep 3.9 (1.8) 3.7 (1.1) 2.4 (1.7) 
G4Y11 Croatia Mathprep | —1.0 (4.3) 3.5 (3.8) 0.0 (4.7) 
G4Y11 Czech Republic Mathprep 5.4 (2.9) 5.5 (2.5) 6.9 (4.8) 
G4Y11 Denmark Mathprep 0.1 (2.0) -0.1 (1.4) —0.1 (2.0) 
G4Y11 Finland Mathprep | —1.1 (3.2) 0.8 (2.5) 0.9 (5.9) 
G4Y11 Georgia Mathprep 0 (3.4) 0.2 (2.8) 2.3 (4.9) 
G4Y11 Germany Mathprep 2.7 (1.7) 3 (1.4) 5.6 (2.8) 
G4Y11 Honduras Mathprep 3.2 (4.3) 4.6 (2.1) 2.7 (5.2) 
G4Y11 Hong Kong Mathprep 6.9 (3.9) 7.8 (3.8) 5.0 (4.2) 
G4Y11 Hungary Mathprep | —2.4 (9.1) | -10.4 (17.2) | -26.2 (12) 
G4Y11 Tran Mathprep 1.0 (2.3) 0.5 (2.0) —2.9 (3.3) 
G4Y11 Treland Mathprep | —1.7 (3.5) —2.2 (2.6) —0.7 (5.0) 
G4Y11 Italy Mathprep 4.0 (4.5) 2.3 (3.9) 3.6 (4.9) 
G4Y11 Japan Mathprep | —1.0 (1.7) —0.6 (1.4) —3.0 (2.3) 
G4Y11 Korea Mathprep 0.7 (4.9) 0.0 (5.1) 0.3 (4.0) 
G4Y11 Kuwait Mathprep | —1.5 (4.8) —-1.6 (3.8) 0.4 (4.9) 
G4Y11 Lithuania Mathprep | —1.4 (4.3) 0.2 (6.2) 0.6 (5.4) 
G4Y11 Malta Mathprep | —0.9 (1.3) —1.1 (1.7) —0.2 (2.7) 
G4yY11 Morocco Mathprep 4.3 (6.2) 11.7 (4.8) 0.5 (6.4) 
G4Y11 Oman Mathprep 0.4 (2.7) 1.7 (2.0) 1.4 (2.6) 
G4Y11 Netherlands Mathprep 0.5 (7.3) 3.2 (4.6) 5.5 (6.4) 
G4Y11 New Zealand Mathprep | —2.8 (3.6) 2.0 (3.1) —1.1 (4.2) 
G4Y11 Norway Mathprep | —1.5 (3.0) -1.1 3.1) —2.8 (3.6) 
G4Y11 Portugal Mathprep | —4.5 (3.8) —5.0 (4.2) —0.5 (6.9) 
G4Y11 Qatar Mathprep 5.9 (6.5) 5.6 (3.8) 3.3 (5.9) 
G4Y11 Romania Mathprep 2.1 (2.8) 2.0 (2.0) 2.2 (4.0) 
G4Y11 Saudi Arabia Mathprep 5.1 (4.3) 4.1 (3.0) 5.2 (4.5) 
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Grade/year | Country Predictor | SLM MLM CLM 

G4yY11 Serbia Mathprep 7.7 (3.1) 3.5 (3.5) —2.0 (4.8) 
G4Y11 Singapore Mathprep 1.4 (2.6) 3.4 (3.5) 1.5 (2.0) 
G4Y11 Slovak Republic Mathprep | —0.9 (5.4) 0.1 (4.8) —4,3 (6.2) 
G4Y11 Spain Mathprep 0.4 (2.9) —2.4 (2.4) -0.4 (3.1) 
G4Y11 Sweden Mathprep | —5.0 (2.9) —3.6 (2.1) —4,3 (2.7) 
G4Y11 Thailand Mathprep 0.8 (4.2) 0.8 (2.7) —1.8 (4.3) 
G4Y11 United Arab Emirates Mathprep | —0.3 (3.1) 2.1 (2.1) 0.6 (3.2) 
G4Y11 Tunisia Mathprep | —2.5 (2.8) —3.4 (2.0) —4.1 (2.8) 
G4Y11 United States Mathprep 1.0 (2.4) 0.2 (1.7) —0.4 (2.4) 
G4Y11 Yemen Mathprep 4.3 (4.1) 3.8 (2.6) 5.9 (5.5) 
G4Y11 England Mathprep | —3.9 (3.2) —3.6 (2.9) -7.9 (4.9) 
G4Y11 Northern Ireland Mathprep 6.5 (5.9) 3.4 (2.7) 10.7 (7.5) 
G4Y11 United Arab Emirates Mathprep 4.4 (2.9) 1.7 (3.0) 4.2 (3.7) 

(Dubai) 
G4Y11 United Arab Emirates (Abu | Mathprep | —5.7 (4.7) -3.7 (4.4) —3.6 (5.4) 
Dhabi) 

G4yY11 Canada (Ontario) Mathprep 3.3 (2.9) 3.0 (2.8) —0.8 (3.4) 
G4Y11 Canada (Quebec) Mathprep 2.7 (2.3) 1.7 (2.6) 0.3 (2.7) 
G4Y11 Canada (Alberta) Mathprep 2.3 (2.9) 2.6 (3.6) 1.1 (3.7) 
G4Y15 Australia Mathprep 4.3 (4.6) 7.7 (3.3) 9.4 (5.3) 
G4yY15 Bahrain Mathprep 6.3 (2.4) 5.8 (3.6) 7A (2.6) 
G4Y15 Bulgaria Mathprep 8.3 (8.7) 0.1 (6.6) 8.6 (9.3) 
G4Y15 Canada Mathprep 3.3 (2.5) 0.5 (1.9) 3.6 (4.6) 
G4yY15 Chile Mathprep 5.6 (6.8) 9.6 (4.7) 3.7 (5.4) 
G4yY15 Chinese Taipei Mathprep | —1.4 (1.8) —-1.5 (1.5) —0.9 (2.1) 
G4Y15 Cyprus Mathprep 2.8 (3.8) 2.1 (2.7) 1.8 (3.7) 
G4Y15 Czech Republic Mathprep 2.4 (2.4) 3.3 (2.0) 3.5 (2.3) 
G4yY15 Denmark Mathprep | —2.3 (2.3) —-1.9 (1.7) 0.4 (2.5) 
G4Y15 Finland Mathprep 2.8 (3.7) 5.0 (3.7) 4.5 (5.6) 
G4Y15 France Mathprep 3.6 (2.3) 3.6 (2.1) 1.7 (2.9) 
G4yY15 Georgia Mathprep | —1.2 (4.3) 4.6 (5.0) —2.3 (7.3) 
G4Y15 Germany Mathprep 5.9 (3.1) 6.2 (1.7) 9.9 (5.0) 
G4Y15 Hong Kong Mathprep 0.8 (4.0) —4,3 (6.5) 5.3 (3.5) 
G4Y15 Hungary Mathprep 7.8 (9.6) 10.9 (5.9) —2.1 (7.9) 
G4Y15 Indonesia Mathprep | —3.6 (3.3) —-1.3 (1.9) —4,3 (3.9) 
G4yY15 Tran Mathprep 1.5 (4.0) 3.4 (2.1) —1.0 (3.8) 
G4Y15 Treland Mathprep | —0.7 (2.3) —-0.2 (1.8) 0.9 (2.6) 
G4yY15 Italy Mathprep 0.1 (3.0) 1.9 3.1) —3.7 (4.0) 
G4Y15 Japan Mathprep 0.6 (2.3) —-0.3 (2.0) 2.1 (3.0) 
G4yY15 Kazakhstan Mathprep 5.4 (3.7) 5.1 (3.2) 6.6 (3.6) 
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Grade/year | Country Predictor | SLM MLM CLM 
G4yY15 Korea Mathprep 1.0 (4.6) 0.9 (2.2) 1.4 (5.2) 
G4Y15 Kuwait Mathprep | —0.1 (4.3) —-1.8 (3.8) 2.7 (4.4) 
G4yY15 Lithuania Mathprep | —4.4 (3.6) 2.4 (3.6) —3.4 (4.3) 
G4yY15 Morocco Mathprep 3.3 (4.4) 4.2 (2.7) 3.2 (5.4) 
G4Y15 Oman Mathprep 4.9 (3.8) 2.8 (2.7) 6.6 (4.4) 
G4Y15 New Zealand Mathprep | —3.2 (3.8) -3.9 (3.4) —0.7 (4.0) 
G4Y15 Norway Mathprep | —1.4 (2.4) —-1.4 (1.8) —2.6 (2.7) 
G4Y15 Poland Mathprep 3.4 (3.5) 3.1 (4.2) 3.2 (3.1) 
G4Y15 Portugal Mathprep —8.8 (4.5) —6.7 (4.0) -7.9 (4.4) 
G4Y15 Qatar Mathprep 3.3 (3.8) 2.8 (3.7) 4.1 (3.9) 
G4Y15 Serbia Mathprep 3.9 (4.6) 3.3 (3.6) 2.9 (6.0) 
G4Y15 Singapore Mathprep 4.6 (3.0) 0.1 (4.6) 1.2 (2.4) 
G4yY15 Slovak Republic Mathprep 6.6 (3.6) 1.4 (3.6) 10.2 (3.9) 
G4yY15 Slovenia Mathprep 1.2 (6.2) —2.7 (5.5) —6.1 (8.2) 
G4Y15 Sweden Mathprep 1.4 (2.3) 1.8 (2.1) 0.2 (2.5) 
G4yY15 United Arab Emirates Mathprep | —10.4 (4.1) 0.3 (2.1) | -11.7 (4.8) 
G4yY15 Turkey Mathprep 5.7 (4.2) 12.3 (3.9) 2.4 (4.0) 
G4Y15 United States Mathprep -1.7 (3.6) —8.4 (4.0) —2.4 (3.0) 
G4yY15 England Mathprep | —2.2 (4.0) 7.5 (6.3) 0.8 (4.2) 
G4Y15 Northern Ireland Mathprep 4.7 (3.3) 4.5 (3.0) 0.0 (4.6) 
G4Y15 United Arab Emirates Mathprep 3.9 (3.9) 3.4 (3.7) 4.9 (4.6) 
(Dubai) 
G4Y15 Canada (Ontario) Mathprep 3.6 (2.9) 0.2 (2.8) 9.7 (3.8) 
G4yY15 Canada (Quebec) Mathprep | —5.4 (4.4) —5.9 (4.2) —5.6 (5.9) 
G4Y15 Argentina (Buenos Aires) Mathprep | —4.6 (9) —3.5 (5.5) 0.9 (4.7) 
G8Y03 Australia Mathprep 6.1 (5.2) 3 (2.8) 5.2 (5.7) 
G8Y03 Bahrain Mathprep 6.9 (2.4) 3.9 (2.8) 8.9 (2.3) 
G8Y03 Botswana Mathprep | —2.3 (2.8) —1.6 (2.3) —3.2 (2.9) 
G8Y03 Bulgaria Mathprep 5.0 (5.9) —5.3 (9.5) 4.9 (6.1) 
G8Y03 Chile Mathprep 8.0 (3.1) 8.2 (2.7) 2.3 (2.4) 
G8Y03 Chinese Taipei Mathprep 6.8 (2.9) 8.3 (2.3) 2.5 (2.2) 
G8Y03 Cyprus Mathprep | —4.9 (3.9) —-7.6 (4.6) —8.0 (4.2) 
G8Y03 Estonia Mathprep 4.2 (2.6) 2.7 (1.4) 3.3 (2.7) 
G8Y03 Palestinian Mathprep | —1.4 (4.1) 0.2 (2.6) —0.5 (4.2) 
G8Y03 Hong Kong Mathprep 0.1 (3.6) 2.7 (4.8) 2.5 (3.4) 
G8Y03 Hungary Mathprep | —1.3 (3.3) 1.5 (3.9) 0.4 (3.2) 
G8Y03 Indonesia Mathprep | —1.8 (6.5) 0.9 (3.7) —4.0 (6.5) 
G8Y03 Tran Mathprep 1.3 (3.5) —0.1 (2.8) 3.5 (3.3) 
G8Y03 Israel Mathprep | —2.6 (4.6) 6.3 (5.0) 6.8 (3.9) 
G8Y03 Japan Mathprep 0.9 (2.1) 0.3 (1.4) 1.9 (1.9) 


(continued) 


192 


Table B.12 (continued) 


Appendix B: Detailed Regression Estimates ... 


Grade/year | Country Predictor | SLM MLM CLM 
G8Y03 Jordan Mathprep 0.9 (4.4) 0.4 (4.1) 0.3 (5.3) 
G8Y03 Lithuania Mathprep | —2.7 (4.4) —8.3 (5.8) —0.1 (4.5) 
G8Y03 Malaysia Mathprep | —0.3 (3.0) 0.0 (2.3) —2.3 (2.4) 
G8Y03 Moldova Mathprep | —2.7 (4.2) -7.3 (5.5) —2.9 (5.1) 
G8Y03 New Zealand Mathprep 7.9 (3.4) 2.2 (3.2) 9.0 (2.8) 
G8Y03 Norway Mathprep | —2.7 (2.5) —2.6 (1.7) —3.0 (3.2) 
G8Y03 Philippines Mathprep 4.3 (6.0) 4.8 (5.1) 6.9 (4.7) 
G8Y03 Romania Mathprep 9.4 (9.2) 20.6 (7.5) 7.5 (1.1) 
G8Y03 Singapore Mathprep 3.6 (3.2) 1.9 (5.6) 2.2 (2.6) 
G8Y03 Slovenia Mathprep | —0.7 (2.6) 1.1 1.7) 0.1 (2.9) 
G8Y03 Sweden Mathprep 1.8 (2.3) 5.7 (3.1) 3.9 (2.3) 
G8Y03 Macedonia Mathprep 4.3 (4.8) 5.8 (3.2) 1.5 (4.8) 
G8Y03 Egypt Mathprep 1.2 (13.5) | —4.5 (9.2) 2.9 (12.3) 
G8Y03 Serbia Mathprep 3.1 GB.1) 3.4 (2.5) 2.4 (2.7) 
G8Y03 Canada (Ontario) Mathprep 0.1 (3.1) 1.0 (2.3) —0.9 (2.6) 
G8Y03 Canada (Quebec) Mathprep 3.7 (3.7) 5.6 (2.2) 1.4 (2.8) 
G8Y07 Algeria Mathprep 0.9 (1.6) 0.7 1.1) 0.4 (1.6) 
G8Y07 Australia Mathprep 7.9 (2.9) 5.8 (2.8) 3.3 (2.6) 
G8Y07 Bahrain Mathprep | —1.6 (2.6) —0.6 (2.8) —0.1 (3.2) 
G8Y07 Armenia Mathprep 1.1 (5.7) —4.1 (5.9) 4.9 (6.0) 
G8Y07 Bosnia and Herzegovina Mathprep 5.1 (5.4) 5.4 (4.0) 6.2 (5.2) 
G8Y07 Botswana Mathprep | —0.9 (3.8) -1.1 3.4) 0.3 (3.1) 
G8Y07 Bulgaria Mathprep 15.7 (9.1) 15 (6.7) 12.8 (12.1) 
G8Y07 Chinese Taipei Mathprep 0.9 (2.7) £3°(25) —3.3 (2.4) 
G8Y07 Colombia Mathprep 9.0 (4.1) 9.8 (2.9) 3.1 (4.2) 
G8Y07 Cyprus Mathprep 4.9 (4.1) 3.4 (3.3) 4.2 (3.9) 
G8Y07 Czech Republic Mathprep 5.0 (1.5) 2.2 (2.3) 2.5 (1.5) 
G8Y07 Georgia Mathprep 0.3 (4.3) 5.0 (3.8) 1.5 (5.3) 
G8Y07 Palestinian Mathprep | —4.1 (5.1) —3.4 (4.8) —5.0 (5.8) 
G8Y07 Ghana Mathprep 1.1 (3.7) 0.1 (2.2) 0.0 (3.5) 
G8Y07 Hong Kong Mathprep | —4.9 (4.8) —-6.0 (5.7) —4.5 (3.7) 
G8Y07 Hungary Mathprep 6.9 (6.5) 13.4 (7.3) 9.2 (5.3) 
G8Y07 Indonesia Mathprep 7.6 (4.5) 7.1 (3.8) 9.5 (4.2) 
G8Y07 Tran Mathprep | —1.4 (3.4) —-1.5 (2.7) —2.4 (3.2) 
G8Y07 Israel Mathprep 1.7 (5.6) —2.6 (5.8) 2.0 (4.8) 
G8Y07 Japan Mathprep 2.0 (3.4) 0.7 (1.2) 2.1 (3.3) 
G8Y07 Jordan Mathprep | —5.8 (6.8) —9.9 (5.0) —-1.3 (7.3) 
G8Y07 Korea Mathprep 1.1 (5.0) 10.2 (7.3) 7.3 (3.9) 
G8Y07 Kuwait Mathprep | —2.9 (3.5) —4.1 (3.5) —0.4 (3.5) 
G8Y07 Lebanon Mathprep | —0.8 (4.2) -3.1 (1.4) —1.0 (3.3) 
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Grade/year | Country Predictor | SLM MLM CLM 
G8Y07 Lithuania Mathprep | —0.4 (2.9) -0.1 (3.4) —2.9 (3.1) 
G8Y07 Malaysia Mathprep | —0.5 (3.4) —3.0 (5.5) 2.7 (3.2) 
G8Y07 Malta Mathprep | —7.0 (1.2) 0.7 (1.5) —4.8 (3.4) 
G8Y07 Oman Mathprep 3.2 (3.8) 3.2 (2.3) —-0.3 (4.1) 
G8Y07 Norway Mathprep | —0.6 (1.8) —-1.8 (1.6) —1.7 (1.7) 
G8Y07 Qatar Mathprep | —2.8 (1.3) 0.7 (1.7) —4.2 (3.5) 
G8Y07 Romania Mathprep | —2.1 (4.6) —-0.4 (5.7) | -10.4 (6.1) 
G8Y07 Saudi Arabia Mathprep | —0.8 (4.2) -1.7 (3.5) 1.9 (4.8) 
G8Y07 Singapore Mathprep 2.7 (3.3) -3.3 (4.1) 2.5 (2.3) 
G8Y07 Slovenia Mathprep | —2.6 (3.2) —3.0 (3.8) —5.0 (3.3) 
G8Y07 Sweden Mathprep 0.3 (1.6) 1.7 (1.8) 0.1 (2.5) 
G8Y07 Syria Mathprep 5.9 (5.4) 6.0 (5.3) 4.3 (6.4) 
G8Y07 Thailand Mathprep | —1.9 (5.0) -1.8 (3.4) 4.3 (4.2) 
G8Y07 Tunisia Mathprep | —4.2 (2.7) —5.5 (2.1) —3.5 (2.2) 
G8Y07 Turkey Mathprep 5.0 (4.9) 10.3 (4.8) —0.5 (4.9) 
G8Y07 Ukraine Mathprep | —2.6 (4.7) 1.4 (4.8) —2.5 (4.3) 
G8Y07 Egypt Mathprep 4.2 (4.5) 7.7 (3.9) 3.9 (4.6) 
G8Y07 United States Mathprep 4.4 (2.0) 3.1 (3.4) 5.9 (2.0) 
G8Y07 Serbia Mathprep 0.6 (4.6) 0.9 (3.5) —0.8 (4.9) 
G8Y07 England Mathprep | —8.7 (3.7) -7.4 (7.5) —6.5 (4.8) 
G8Y07 Scotland Mathprep | —4.7 (3.9) —3.6 (5.6) —4.6 (3.8) 
G8Y07 Spain (Basque) Mathprep 3.6 (3.4) 4.9 (4.2) 4.8 (3.3) 
G8Y07 United Arab Emirates Mathprep 19.1 (12.1) 16.9 (15.3) 28.8 (12.5) 
(Dubai) 
G8Y07 Canada (British Columbia) | Mathprep 5.6 (2.6) 5.7 (3.6) 6.4 (3.4) 
G8Y07 United States Mathprep 5.5 (4.1) 4.2 (5.3) —0.8 (3.3) 
(Massachusetts) 
G8Y07 United States (Minnesota) Mathprep 7.1 (6.0) 6.1 (9.9) 1.2 (6.0) 
G8Y11 Australia Mathprep 1.1 (2.8) 0.9 (3.3) 0.7 (2.6) 
G8Y11 Bahrain Mathprep 14.4 (3.4) 1.8 (6.3) 0.6 (3.8) 
G8Y11 Botswana Mathprep | —0.6 (3.1) —0.4 (2.0) —3.7 (2.4) 
G8Y11 Chile Mathprep 8.9 3.1) 13.2 (2.7) 3.8 (2.7) 
G8Y11 Georgia Mathprep 0.9 (4.7) 0.0 (2.9) 6.0 (5.5) 
G8Y11 Palestinian Mathprep | —7.8 (4.1) —6.8 (2.7) —-7.6 (4.5) 
G8Y11 Honduras Mathprep | —3.1 (3.2) 1.1 (2.2) —3.0 (3.2) 
G8Y11 Hong Kong Mathprep | —5.6 (3.9) —8.2 (5.1) —4.2 (3.1) 
G8Y11 Hungary Mathprep 11.0 (4.2) 5.6 (4.2) 7.2 (3.6) 
G8Y11 Indonesia Mathprep 4.7 (4.7) 3.8 (4.9) 1.4 (3.7) 
G8Y11 Tran Mathprep 0.8 (4.7) —-1.0 (3.4) —2.2 (4.0) 
G8Y11 Israel Mathprep 11.1 (4.3) 7.3 (3.6) 6.5 (4.1) 
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Grade/year | Country Predictor | SLM MLM CLM 

G8Y11 Jordan Mathprep 3.0 (5.6) —0.8 (4.3) 2.5 (6.2) 
G8Y11 Korea Mathprep 3.8 (4.3) 0.7 (5.6) 3.9 (4.8) 
G8Y11 Lebanon Mathprep | —0.5 (3.0) 1.7 (2.5) 0.0 (2.9) 
G8Y11 Lithuania Mathprep 2.6 (2.8) 3.1 (2.5) 4.9 (2.9) 
G8Y11 Malaysia Mathprep | —8.1 (3.7) —8.0 (3.4) —4,3 (2.8) 
G8Y11 Morocco Mathprep | —2.7 (3.7) —1.0 (2.7) -7.0 (2.9) 
G8Y11 Oman Mathprep | —2.0 (2.3) -3.7 (2.1) -1.9 (2.4) 
G8Y11 New Zealand Mathprep 6.6 (3.7) 12.4 (4.6) 5.4 (3.0) 
G8Y11 Norway Mathprep 1.7 (1.8) 1.3 (1.2) 0.7 (1.8) 
G8Y11 Romania Mathprep | —7.6 (4.0) —-1.8 (4.7) —4.3 (4.1) 
G8Y11 Saudi Arabia Mathprep 5.4 (6.5) 4.2 (5.5) 3.1 (7.2) 
G8Y11 Singapore Mathprep 3.9 (2.8) 0.7 (5.1) 1.9 (2.2) 
G8Y11 Slovenia Mathprep 1.2 (1.8) 1.7 (2.1) 0.1 (2.0) 
G8Y11 South Africa Mathprep 0.0 (2.7) -7.4 (9.3) -1.9 3.1) 
G8Y11 Sweden Mathprep 0.7 (1.7) 0.7 (1.8) 3.7 (1.7) 
G8Y11 Syria Mathprep | —1.5 (5.7) —2.0 (3.2) —0.6 (5.2) 
G8Y11 Thailand Mathprep 0.5 (4.2) 1.5 (4.0) 4.2 (3.9) 
G8Y11 Tunisia Mathprep 0.4 (4.3) 2.4 (2.8) 0.6 (2.7) 
G8Y11 Turkey Mathprep | —1.7 (4.9) —2.1 (3.7) 0.5 (4.2) 
G8Y11 Ukraine Mathprep | —4.2 (4.1) 3:1. 2:5) —-7.7 (5.6) 
G8Y11 Macedonia Mathprep 5.0 (6.1) 18.9 (10.2) 5.9 (5.7) 
G8Y11 United States Mathprep 3.6 (2.3) 1.4 (6.8) 4.5 (2.4) 
G8Y11 England Mathprep | —1.0 (4.4) —-1.5 (5.5) —0.3 (3.8) 
G8Y11 United Arab Emirates Mathprep 10.3 (3.7) 1.8 (6.7) 6.8 (3.7) 

(Dubai) 
G8Y11 United Arab Emirates (Abu | Mathprep 2.2 (4.6) 1.9 (3.6) -1.8 (4.1) 
Dhabi) 

G8Y11 Canada (Ontario) Mathprep 2.8 (2.4) 1.3 (3.9) —-1.9 (3.8) 
G8Y11 Canada (Quebec) Mathprep 3.6 (2.1) 4.3 (6.3) 3.5 (2.3) 
G8Y11 Canada (Alberta) Mathprep | —0.3 (1.6) 2.6 (3.1) 1.6 (1.8) 
G8Y15 Australia Mathprep 4.0 (2.5) 12.1 (5.9) 5.7 (1.9) 
G8Y15 Bahrain Mathprep 6.3 (2.7) 5.3 (3.0) 7.7 (2.9) 
G8Y15 Botswana Mathprep | —0.1 (2.7) 0.6 (2.4) —1.8 (2.5) 
G8Y15 Canada Mathprep 7.3 (2.0) 1.9 (2.4) 7.5 (2.6) 
G8Y15 Chile Mathprep 9.0 (3.4) 11.1 (3.4) 5.1 (3.2) 
G8Y15 Chinese Taipei Mathprep 6.3 (2.0) 4.3 (4.5) 0.2 (4.8) 
G8Y15 Georgia Mathprep 0.3 (4.8) —0.6 (4.3) 0.1 (7.8) 
G8Y15 Hong Kong Mathprep | —8.9 (3.3) -1.0 4.1) —6.3 (3.6) 
G8Y15 Hungary Mathprep | —3.7 (3.7) —-1.8 (5.5) —3.5 (3.9) 
G8Y15 Tran Mathprep 9.1 (3.6) 6.5 (3.1) 6.1 3.1) 
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Predictor | SLM MLM CLM 

Mathprep 3.0 (2.7) 4.7 (2.3) 3.0 (2.2) 
Mathprep 3.4 (4.2) —0.7 (4.0) 3.8 (3.0) 
Mathprep | 0.3 (1.7) | -12(1.7) | 04 (1.6) 
Mathprep | —0.3 (2.5) 0.5 (2.2) 3.0 (2.6) 
Mathprep 20.6 (7.9) 7.3 (9.8) 20.2 (7.5) 
Mathprep | —S.7 (3.9) —4,.2 (2.5) —5.9 (3.5) 
Mathprep 1.3 (2.9) 0.6 (4.7) —2.4 (3.4) 
Mathprep 7.1 (6.5) 3.4 (2.8) 11.9 (4.3) 
Mathprep 4.7 (2.7) 4.2 (2.0) 2.0 (2.4) 
Mathprep | —0.3 (3.7) 7.1 (4.3) 1.6 (4.1) 
Mathprep 5.6 (3.7) 0.1 (9.2) 4.3 (2.4) 
Mathprep | —5S.6 (1.4) —2.5 (1.7) 5.0 (4.8) 
Mathprep | —4.6 (3.8) —5.8 (3.5) —3.3 (3.8) 
Mathprep 3.3 (2.7) 3.4 (3.8) 3.9 (2.3) 
Mathprep 0.2 (1.5) —-1.0 (1.3) —2.2 (1.9) 
Mathprep 0.2 (4.2) —8.0 (6.3) 4.5 (4.0) 
Mathprep 4.6 (2.9) 6.0 (4.2) 2.7 (2.4) 
Mathprep | —3.8 (2.7) | -1.9(2.5) | 2.1 (2.8) 
Mathprep 1.1 4.1) —1.8 (1.8) 2.1 (3.6) 
Mathprep 3.8 (2.4) 4.1 (3.0) 3.3 (2.3) 
Mathprep | —9.6 (3.4) —5.9 (3.4) —9.0 (3.8) 
Mathprep 12.8 (4.6) -3.1 (3.9) 12.8 (3.7) 
Mathprep 1.8 (3.2) 2.3 (2.5) —1.1 (2.9) 


Grade/year | Country 
G8Y15 Ireland 
G8Y15 Israel 
G8Y15 Italy 

G8Y15 Japan 
G8Y15 Kazakhstan 
G8Y15 Jordan 
G8Y15 Korea 
G8Y15 Kuwait 
G8Y15 Lebanon 
G8Y15 Lithuania 
G8Y15 Malaysia 
G8Y15 Malta 
G8Y15 Oman 
G8Y15 New Zealand 
G8Y15 Norway 
G8Y15 Qatar 
G8Y15 Singapore 
G8Y15 Slovenia 
G8Y15 South Africa 
G8Y15 Sweden 
G8Y15 Thailand 
G8Y15 United Arab Emirates 
G8Y15 Turkey 


G8Y15 Egypt Mathprep 1.2 (4.2) 4.5 (3.0) 2.2 (4.3) 

G8Y15 United States Mathprep | —0.4 (2.2) 2.2 (3.8) 1.6 (2.1) 

G8Y15 England Mathprep 7.3 (4.1) 4.1 (9.6) 5.6 (3.0) 

G8Y15 United Arab Emirates Mathprep 2.4 (3.9) -6.1 (5.1) 3.4 (3.7) 
(Dubai) 

G8Y15 United Arab Emirates (Abu | Mathprep 11.1 (9.0) —3.4 (8.0) 15.4 (6.1) 
Dhabi) 

G8Y15 Canada (Ontario) Mathprep | —1.6 (3.0) -3.8 (3.1) —3.3 (3.5) 

G8Y15 Canada (Quebec) Mathprep 7.1 (3.0) 4.3 (5.2) 7A (2.5) 

G8Y15 Argentina (Buenos Aires) Mathprep | —9.4 (7.8) —8.1 (5.0) —4.6 (7.3) 


Notes Standard errors in parentheses. Grade/year indicates grade level (where G4 and G8 are grade four 
and eight, respectively) and year (where e.g., Y07 is the year 2007), resulting in a composite abbreviation 
such as G4Y07 (which indicates Grade 4, 2007 cycle of TIMSS data). SLM = single level model, MLM 
= multilevel model, CLM = classroom-level model, Mathprep = index (1-5) of teacher education to teach 
mathematics 
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Table B.13 Countries where the relationship of time teaching mathematics (Mathtime) with 
student achievement was not statistically significant in all three models 


Grade/year | Country Predictor | SLM MLM CLM 

G4Y03 Chinese Taipei Mathtime 0.0 (0.0) 0.0 (0.0) | 0.0 (0.1) 
G4Y03 Cyprus Mathtime 0.1 (0.1) 0.1 (0.1) 0.0 (0.1) 
G4Y03 Hong Kong Mathtime 0.0 (0.0) 0.0 (0.1) | —0.1 (0.1) 
G4Y03 Tran Mathtime | 0.1 (0.1) | 0.1(0.1) | 0.0 (0.1) 
G4Y03 Japan Mathtime 0.1 (0.1) 0.1 (0.0) 0.1 (0.1) 
G4Y03 Netherlands Mathtime 0.0 (0.1) 0.0 (0.1) | 0.0 (0.1) 
G4Y03 New Zealand Mathtime 0.0 (0.0) 0.0 (0.0) | 0.0 (0.1) 
G4Y03 Singapore Mathtime 0.2 (0.2) 0.0 (0.2) 0.2 (0.2) 
G4Y03 Slovenia Mathtime | —0.1 (0.1) | -0.1 (0.1) | —0.2 (0.1) 
G4Y03 Yemen Mathtime 0.1 (0.1) 0.1 (0.0) 0.1 (0.1) 
G4Y03 Canada (Ontario) Mathtime | —0.1 (0.1) 0.0 (0.0) | —0.1 (0.1) 
G4Y07 Algeria Mathtime |—0.1 (0.1) | —0.1 (0.1) | -0.1 (0.1) 
G4Y07 Australia Mathtime 0.0 (0.1) 0.0 (0.0) | 0.0 (0.1) 
G4Y07 Austria Mathtime 0.1 (0.1) 0.1 (0.1) 0.2 (0.1) 
G4yY07 Armenia Mathtime | 0.1 (0.1) | 0.00.0) | 0.1 (0.1) 
G4Y07 Chinese Taipei Mathtime 0.0 (0.1) 0.0 (0.0) | 0.0 (0.1) 
G4yY07 Colombia Mathtime | 0.0 (0.1) | —-0.1 (0.1) | 0.0 (0.1) 
G4Y07 Denmark Mathtime | —0.1 (0.1) | -0.1 (0.1) | -0.1 (0.1) 
G4Y07 El Salvador Mathtime 0.0 (0.1) 0.1. (0.1) | 0.0 (0.1) 
G4Y07 Georgia Mathtime | —0.1 (0.1) 0.0 (0.1) | —0.2 (0.2) 
G4Y07 Germany Mathtime 0.0 (0.1) 0.0 (0.0) | 0.0 (0.1) 
G4yY07 Tran Mathtime | 0.0 (0.1) | 0.0 (0.0) | 0.0 (0.1) 
G4Y07 Japan Mathtime 0.0 (0.1) 0.0 (0.1) | 0.1 (0.1) 
G4Y07 Lithuania Mathtime 0.1 (0.1) 0.1 (0.1) 0.1 (0.1) 
G4Y07 Mongolia Mathtime 0.1 (0.1) 0.0 (0.1) 0.1 (0.1) 
G4Y07 Singapore Mathtime | —0.2 (0.2) 0.1 (0.2) 0.0 (0.1) 
G4Y07 Slovenia Mathtime | —0.1 (0.1) 0.0 (0.1) | 0.0 (0.1) 
G4yY07 Sweden Mathtime | 0.0 (0.0) | 0.0 (0.0) | 0.0 (0.0) 
G4Y07 Tunisia Mathtime 0.0 (0.1) 0.0 (0.1) | 0.0 (0.1) 
G4Y07 Yemen Mathtime | —0.1 (0.1) | —0.1 (0.1) | -0.1 (0.1) 
G4Y07 England Mathtime 0.1 (0.1) 0.0 (0.0) | 0.0 (0.1) 
G4Y07 Scotland Mathtime 0.0 (0.1) | -0.1 (0.0) | 0.0 (0.1) 
G4Y07 Canada (Ontario) Mathtime 0.0 (0.1) 0.10.1) | 0.0 (0.1) 
G4Y07 Canada (Quebec) Mathtime 0.0 (0.0) 0.1 (0.0) | 0.0 (0.0) 
G4Y07 Canada (Alberta) Mathtime 0.0 (0.1) 0.0 (0.1) | 0.0 (0.1) 
G4Y07 United States (Minnesota) Mathtime 0.0 (0.1) 0.0 (0.1) | 0.0 (0.1) 
G4Y11 Australia Mathtime 0.0 (0.0) 0.0 (0.0) 0.0 (0.0) 
G4Y11 Botswana Mathtime 0.0 (0.0) 0.0 (0.0) 0.0 (0.0) 
G4Y11 Chile Mathtime | 0.0 (0.0) | 0.0 (0.0) | 0.0 (0.0) 
G4yY11 Chinese Taipei Mathtime 0.0 (0.0) 0.0 (0.0) 0.0 (0.0) 
G4Y11 Croatia Mathtime | 0.0 (0.0) | 0.0 (0.0) | 0.0 (0.0) 
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Grade/year | Country Predictor | SLM MLM CLM 
G4Y11 Czech Republic Mathtime 0.0 (0.0) 0.0 (0.0) 0.0 (0.0) 
G4Y11 Denmark Mathtime | —0.1 (0.1) | -0.1 (0.1) | 0.0 (0.1) 
G4Y11 Finland Mathtime | 0.0 (0.0) | 0.0 (0.1) | 0.0 (0.1) 
G4Y11 Georgia Mathtime 0.0 (0.1) 0.0 (0.1) | —0.1 (0.1) 
G4Y11 Germany Mathtime 0.0 (0.0) 0.0 (0.0) | 0.0 (0.1) 
G4Y11 Honduras Mathtime 0.0 (0.0) 0.0 (0.0) | 0.0 (0.1) 
G4Y11 Hong Kong Mathtime 0.0 (0.1) 0.0 (0.1) | 0.0 (0.1) 
G4yY11 Treland Mathtime | 0.0 (0.0) | 0.0 (0.0) | 0.0 (0.0) 
G4Y11 Italy Mathtime | 0.0 (0.0) | 0.0 (0.0) | 0.0 (0.0) 
G4Y11 Kuwait Mathtime 0.0 (0.1) 0.0 (0.1) | 0.0 (0.1) 
G4Y11 Lithuania Mathtime | —0.1 (0.0) 0.0 (0.0) 0.0 (0.0) 
G4Y11 Malta Mathtime | 0.0 (0.0) | 0.0 (0.0) | 0.0 (0.0) 
G4Y11 Oman Mathtime 0.0 (0.0) 0.0 (0.0) | 0.0 (0.1) 
G4Y11 Netherlands Mathtime 0.0 (0.0) 0.0 (0.0) 0.0 (0.0) 
G4Y11 New Zealand Mathtime 0.1 (0.1) 0.0 (0.1) | 0.0 (0.1) 
G4Y11 Norway Mathtime 0.0 (0.1) 0.0 (0.0) | 0.0 (0.1) 
G4Y11 Portugal Mathtime 0.0 (0.0) 0.0 (0.0) | 0.0 (0.1) 
G4Y11 Qatar Mathtime | 0.0 (0.0) | 0.0 (0.0) | 0.0 (0.0) 
G4Y11 Romania Mathtime 0.1 (0.1) 0.1 (0.1) 0.1 (0.1) 
G4Y11 Saudi Arabia Mathtime 0.0 (0.1) 0.0 (0.1) | 0.0 (0.1) 
G4Y11 Serbia Mathtime | 0.1 (0.1) | 0.1(0.1) | 0.1 (0.1) 
G4Y11 Singapore Mathtime 0.0 (0.0) 0.0 (0.1) 0.0 (0.0) 
G4Y11 Slovak Republic Mathtime 0.8 (0.7) | —0.4 (0.5) 0.9 (0.5) 
G4Y11 Spain Mathtime 0.0 (0.1) 0.0 (0.0) | 0.0 (0.1) 
G4Y11 Sweden Mathtime | 0.0 (0.0) | 0.0 (0.0) | 0.0 (0.0) 
G4Y11 Thailand Mathtime | 0.0 (0.1) | 0.0 (0.0) | 0.0 (0.0) 
G4Y11 Tunisia Mathtime 0.0 (0.1) 0.0 (0.1) | 0.0 (0.1) 
G4Y11 United States Mathtime 0.0 (0.0) 0.0 (0.0) 0.0 (0.0) 
G4Y11 Yemen Mathtime 0.0 (0.0) 0.0 (0.0) | 0.0 (0.1) 
G4Y11 England Mathtime | —-0.1 (0.1) | -0.1 (0.1) | 0.0 (0.1) 
G4Y11 Northern Ireland Mathtime 0.0 (0.0) 0.0 (0.0) 0.0 (0.0) 
G4yY11 Canada (Ontario) Mathtime 0.0 (0.0) 0.0 (0.0) 0.0 (0.0) 
G4Y11 Canada (Quebec) Mathtime 0.0 (0.0) 0.0 (0.0) 0.0 (0.0) 
G4Y11 Canada (Alberta) Mathtime 0.0 (0.0) 0.0 (0.0) 0.1 (0.0) 
G4Y15 Australia Mathtime | —0.1 (0.0) 0.0 (0.0) 0.0 (0.0) 
G4Y15 Bulgaria Mathtime | —0.1 (0.1) 0.0 (0.1) | 0.0 (0.1) 
G4yY15 Cyprus Mathtime | 0.0 (0.0) | 0.0 (0.0) | 0.0 (0.0) 
G4Y15 Czech Republic Mathtime 0.0 (0.0) 0.0 (0.0) 0.0 (0.0) 
G4yY15 Finland Mathtime | 0.0 (0.0) | 0.0 (0.0) | 0.1 (0.0) 
G4yY15 France Mathtime 0.1 (0.1) 0.0 (0.1) 0.1 (0.1) 
G4Y15 Georgia Mathtime 0.0 (0.1) 0.0 (0.1) | 0.0 (0.1) 
G4yY15 Germany Mathtime | —0.1 (0.1) | —0.1 (0.0) | 0.0 (0.1) 
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Grade/year | Country Predictor | SLM MLM CLM 

G4yY15 Hong Kong Mathtime | —0.1 (0.1) 0.0 (0.1) | 0.0 (0.0) 
G4yY15 Hungary Mathtime 0.1 (0.1) 0.1 (0.1) 0.0 (0.1) 
G4yY15 Indonesia Mathtime 0.0 (0.1) 0.0 (0.0) | 0.0 (0.1) 
G4yY15 Tran Mathtime | 0.0 (0.1) | 0.10.1) | 0.0 (0.1) 
G4yY15 Treland Mathtime | —0.1 (0.1) 0.0 (0.0) | 0.0 (0.1) 
G4yY15 Italy Mathtime | 0.0 (0.0) | —-0.1 (0.1) | 0.0 (0.0) 
G4Y15 Kazakhstan Mathtime 0.0 (0.1) | -0.1 0.1) | 0.0 (0.1) 
G4Y15 Korea Mathtime 0.0 (0.1) 0.0 (0.1) | —0.2 (0.1) 
G4Y15 Lithuania Mathtime 0.0 (0.1) 0.0 (0.0) | 0.1 (0.1) 
G4yY15 Morocco Mathtime 0.0 (0.1) 0.0 (0.0) | 0.1 (0.1) 
G4yY15 Oman Mathtime | 0.0 (0.0) | 0.0 (0.0) | 0.0 (0.0) 
G4Y15 New Zealand Mathtime 0.0 (0.0) 0.0 (0.0) 0.0 (0.0) 
G4Y15 Norway Mathtime 0.0 (0.0) 0.0 (0.0) | 0.0 (0.1) 
G4yY15 Portugal Mathtime 0.0 (0.1) 0.0 (0.0) | 0.0 (0.1) 
G4yY15 Qatar Mathtime 0.0 (0.1) 0.0 (0.1) | 0.0 (0.1) 
G4yY15 Serbia Mathtime | 0.1 (0.1) | 0.00.1) | 0.0 (0.1) 
G4Y15 Singapore Mathtime | —0.1 (0.1) | —0.1 (0.1) 0.0 (0.1) 
G4Y15 Slovak Republic Mathtime 0.0 (0.0) 0.0 (0.0) | 0.0 (0.1) 
G4yY15 Slovenia Mathtime 0.0 (0.1) 0.0 (0.1) | 0.0 (0.1) 
G4yY15 Turkey Mathtime 0.0 (0.1) 0.0 (0.1) | 0.0 (0.0) 
G4yY15 United States Mathtime | 0.0 (0.0) | 0.0 (0.0) | 0.0 (0.0) 
G4yY15 England Mathtime 0.0 (0.0) 0.0 (0.1) | 0.0 (0.0) 
G4Y15 United Arab Emirates (Dubai) Mathtime 0.1 (0.1) 0.1 (0.1) 0.0 (0.1) 
G4Y15 Canada (Ontario) Mathtime 0.0 (0.0) 0.0 (0.0) 0.0 (0.0) 
G4yY15 Canada (Quebec) Mathtime 0.0 (0.0) 0.0 (0.0) | 0.0 (0.0) 
G4Y15 Argentina (Buenos Aires) Mathtime 0.0 (0.1) 0.0 (0.1) | 0.0 (0.0) 
G8Y03 Australia Mathtime 0.0 (0.2) 0.0 (0.0) | 0.0 (0.1) 
G8Y03 Bahrain Mathtime | 0.0 (0.0) | —0.1 (0.1) | -0.1 (0.0) 
G8Y03 Botswana Mathtime 0.0 (0.1) 0.0 (0.0) | —0.1 (0.1) 
G8Y03 Bulgaria Mathtime | —0.1 (0.2) 0.0 (0.1) 0.0 (0.2) 
G8Y03 Chile Mathtime | 0.1 (0.1) | 0.0 (0.0) | 0.0 (0.0) 
G8Y03 Cyprus Mathtime 0.4 (0.2) 0.2 (0.3) | 0.4 (0.2) 
G8Y03 Estonia Mathtime 0.0 (0.1) 0.0 (0.1) | 0.0 (0.1) 
G8Y03 Palestinian Mathtime 0.1 (0.1) 0.1 (0.1) 0.1 (0.1) 
G8Y03 Hong Kong Mathtime 0.0 (0.1) 0.1 (0.0) | 0.0 (0.0) 
G8Y03 Hungary Mathtime | —0.1 (0.1) 0.0 (0.1) 0.0 (0.1) 
G8Y03 Indonesia Mathtime 0.1 (0.1) 0.1 (0.1) 0.0 (0.1) 
G8Y03 Israel Mathtime 0.0 (0.1) 0.0 (0.1) | 0.0 (0.1) 
G8Y03 Japan Mathtime 0.2 (0.2) 0.2 (0.1) 0.2 (0.2) 
G8Y03 Malaysia Mathtime 0.1 (0.2) 0.1 (0.1) 0.1 (0.1) 
G8Y03 Moldova Mathtime | —0.1 (0.1) | —0.1 (0.1) | -0.1 (0.1) 
G8Y03 New Zealand Mathtime | —0.1 (0.1) 0.0 (0.1) 0.1 (0.1) 
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Table B.13 (continued) 

Grade/year | Country Predictor | SLM MLM CLM 
G8Y03 Norway Mathtime 0.0 (0.1) 0.0 (0.0) | 0.0 (0.1) 
G8Y03 Philippines Mathtime 0.0 (0.1) | —0.1 (0.1) 0.1 (0.1) 
G8Y03 Romania Mathtime 0.1 (0.1) 0.0 (0.1) 0.0 (0.1) 
G8Y03 Slovenia Mathtime 0.0 (0.1) 0.0 (0.1) | 0.0 (0.1) 
G8Y03 Egypt Mathtime | 0.0 (0.1) | 0.0 (0.1) | 0.0 (0.1) 
G8Y03 Serbia Mathtime | 0.0 (0.1) | —0.1 (0.2) | 0.3 (0.2) 
G8Y03 Canada (Ontario) Mathtime 0.0 (0.0) 0.0 (0.0) | 0.0 (0.0) 
G8Y07 Algeria Mathtime | 0.0 (0.0) | 0.0 (0.0) | 0.0 (0.0) 
G8Y07 Armenia Mathtime | 0.0 (0.0) | 0.0 (0.0) | 0.0 (0.0) 
G8Y07 Botswana Mathtime 0.1 (0.1) 0.1 (0.1) 0.0 (0.1) 
G8Y07 Colombia Mathtime | 0.0 (0.0) | 0.0 (0.0) | 0.0 (0.0) 
G8Y07 Cyprus Mathtime | —0.3 (0.2) | —0.2 (0.2) | —0.3 (0.2) 
G8Y07 Georgia Mathtime 0.2 (0.3) 0.0 (0.2) 0.4 (0.4) 
G8Y07 Palestinian Mathtime 0.0 (0.1) 0.0 (0.0) | 0.0 (0.1) 
G8Y07 Ghana Mathtime | —0.1 (0.0) | —0.1 (0.0) | 0.0 (0.0) 
G8Y07 Indonesia Mathtime 0.0 (0.1) 0.0 (0.1) | 0.0 (0.1) 
G8Y07 Israel Mathtime 0.0 (0.1) 0.3 (0.1) | 0.0 (0.1) 
G8Y07 Japan Mathtime 0.2 (0.2) 0.1 (0.1) 0.1 (0.2) 
G8Y07 Korea Mathtime | —0.1 (0.1) 0.1 (0.2) 0.1 (0.1) 
G8Y07 Kuwait Mathtime | 0.0 (0.0) | 0.0 (0.0) | 0.0 (0.0) 
G8Y07 Lebanon Mathtime 0.0 (0.1) 0.0 (0.0) | 0.0 (0.1) 
G8Y07 Lithuania Mathtime 0.2 (0.1) 0.1 (0.2) 0.3 (0.2) 
G8Y07 Malaysia Mathtime 0.0 (0.2) | —0.4 (0.5) | —0.1 (0.2) 
G8Y07 Oman Mathtime | 0.1 (0.0) | 0.1 (0.0) | 0.0 (0.0) 
G8Y07 Norway Mathtime 0.0 (0.1) 0.0 (0.0) | 0.0 (0.1) 
G8Y07 Romania Mathtime 0.0 (0.1) 0.0 (0.1) | —0.1 (0.1) 
G8Y07 Saudi Arabia Mathtime 0.0 (0.1) 0.0 (0.1) | 0.0 (0.1) 
G8Y07 Slovenia Mathtime | —0.3 (0.2) | —0.4 (0.2) | —0.3 (0.2) 
G8Y07 Syria Mathtime | 0.1 (0.1) | 0.1 (0.0) | 0.1 (0.1) 
G8Y07 Thailand Mathtime | 0.0 (0.1) | 0.0 (0.1) | 0.1 (0.1) 
G8Y07 Tunisia Mathtime 0.0 (0.1) 0.0 (0.1) | 0.0 (0.1) 
G8Y07 Egypt Mathtime 0.1 (0.1) 0.1 (0.0) 0.1 (0.1) 
G8Y07 United States Mathtime 0.0 (0.0) 0.0 (0.1) | 0.0 (0.0) 
G8Y07 Serbia Mathtime | —0.2 (0.2) | 0.0 (0.1) | 0.0 (0.2) 
G8Y07 Spain (Basque) Mathtime 0.2 (0.1) 0.2 (0.1) 0.1 (0.1) 
G8Y07 United Arab Emirates (Dubai) Mathtime 0.1 (0.1) 0.0 (0.1) 0.0 (0.1) 
G8Y11 Botswana Mathtime 0.0 (0.0) 0.0 (0.0) 0.0 (0.0) 
G8Y11 Finland Mathtime | 0.1 0.1) | 0.1 (0.1) | -0.2 (0.1) 
G8Y11 Palestinian Mathtime 0.0 (0.0) 0.0 (0.0) | 0.0 (0.1) 
G8Y11 Ghana Mathtime | 0.0 (0.0) | 0.0 (0.0) | 0.0 (0.0) 
G8Y11 Honduras Mathtime 0.0 (0.1) 0.0 (0.0) | 0.0 (0.1) 
G8Y11 Hungary Mathtime 0.1 (0.1) 0.0 (0.1) | 0.0 (0.1) 
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Grade/year | Country Predictor | SLM MLM CLM 

G8Y11 Indonesia Mathtime 0.0 (0.0) 0.0 (0.0) 0.0 (0.0) 
G8Y11 Italy Mathtime | 0.0 (0.0) | 0.0 (0.0) | 0.0 (0.0) 
G8Y11 Lebanon Mathtime 0.0 (0.1) 0.0 (0.0) | 0.0 (0.1) 
G8Y11 Lithuania Mathtime | 0.1 (0.1) | 0.0 (0.0) | 0.1 (0.1) 
G8Y11 Malaysia Mathtime | —0.1 (0.1) | —0.1 (0.1) 0.0 (0.0) 
G8Y11 Oman Mathtime | 0.0 (0.0) | 0.0 (0.0) | 0.0 (0.0) 
G8Y11 Norway Mathtime | 0.0 (0.0) | 0.0 (0.0) | 0.0 (0.0) 
G8Y11 Romania Mathtime 0.1 (0.1) 0.1 (0.1) 0.1 (0.1) 
G8Y11 Saudi Arabia Mathtime 0.0 (0.1) 0.0 (0.0) | 0.1 (0.1) 
G8Y11 Slovenia Mathtime 0.0 (0.1) 0.1.0.1) | 0.0 (0.1) 
G8Y11 South Africa Mathtime | 0.0 (0.0) | —-0.1 (0.1) | 0.0 (0.0) 
G8Y11 Sweden Mathtime | 0.0 (0.0) | 0.0 (0.0) | 0.0 (0.1) 
G8Y11 Turkey Mathtime | —0.1 (0.1) | —0.1 (0.1) | -0.2 (0.1) 
G8Y11 Macedonia Mathtime 0.0 (0.1) 0.0 (0.1) | 0.1 (0.1) 
G8Y11 United Arab Emirates (Abu Dhabi) | Mathtime 0.1 (0.0) 0.0 (0.0) 0.0 (0.0) 
G8Y11 Canada (Ontario) Mathtime 0.0 (0.0) 0.0 (0.1) 0.0 (0.0) 
G8Y11 Canada (Quebec) Mathtime 0.0 (0.0) 0.1 (0.1) 0.0 (0.0) 
G8Y15 Australia Mathtime 0.0 (0.1) 0.0 (0.1) 0.0 (0.0) 
G8Y15 Bahrain Mathtime 0.0 (0.0) 0.0 (0.1) 0.0 (0.0) 
G8Y15 Botswana Mathtime 0.0 (0.1) 0.0 (0.1) | 0.0 (0.1) 
G8Y15 Chile Mathtime | 0.0 (0.0) | 0.0 (0.0) | 0.0 (0.0) 
G8Y15 Hong Kong Mathtime 0.0 (0.1) 0.0 (0.1) | 0.0 (0.1) 
G8Y15 Ireland Mathtime | —0.1 (0.2) | —-0.2 (0.2) | 0.0 (0.1) 
G8Y15 Italy Mathtime | 0.0 (0.0) | 0.1 (0.0) | 0.0 (0.0) 
G8Y15 Kazakhstan Mathtime 0.1 (0.1) 0.1 (0.1) 0.1 (0.1) 
G8Y15 Korea Mathtime 0.0 (0.1) 0.0 (0.1) | 0.0 (0.1) 
G8Y15 Kuwait Mathtime |—0.1 (0.1) | —-0.1 (0.0) | -0.1 (0.1) 
G8Y15 Lebanon Mathtime 0.0 (0.1) 0.0 (0.0) | —0.1 (0.1) 
G8Y15 Lithuania Mathtime 0.1 (0.1) 0.1 (0.1) 0.2 (0.1) 
G8Y15 Malaysia Mathtime 0.1 (0.1) 0.1 (0.2) 0.1 (0.1) 
G8Y15 Oman Mathtime 0.0 (0.1) 0.0 (0.1) | 0.0 (0.1) 
G8Y15 New Zealand Mathtime 0.1 (0.1) | —0.1 (0.1) 0.1 (0.1) 
G8Y15 Norway Mathtime | —0.1 (0.0) 0.0 (0.0) | —0.1 (0.0) 
G8Y15 Sweden Mathtime 0.0 (0.1) | -0.1 (0.1) | -0.1 (0.1) 
G8Y15 Thailand Mathtime |-0.1 (0.1) |-0.2 (0.1) | 0.0 (0.1) 
G8Y15 United States Mathtime | —0.1 (0.0) | —0.1 (0.1) | 0.0 (0.0) 
G8Y15 United Arab Emirates (Abu Dhabi) | Mathtime 0.0 (0.1) | —0.1 (0.1) 0.0 (0.1) 
G8Y15 Argentina (Buenos Aires) Mathtime 0.0 (0.2) 0.1 (0.1) 0.1 (0.1) 


Notes Standard errors in parentheses. Grade/year indicates grade level (where G4 and G8 are grade four 
and eight, respectively) and year (where e.g., Y07 is the year 2007), resulting in a composite abbreviation 
such as G4Y07 (which indicates Grade 4, 2007 cycle of TIMSS data). SLM = single level model, MLM 
= multilevel model, CLM = classroom-level model, Mathtime = mean number of minutes spent teaching 
mathematics per week 
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Table B.14 Countries where the relationship of alignment of teacher content coverage with 
national expectations (Alignment) with student achievement was not statistically significant in all 


three models 


Grade/year | Country Predictor SLM MLM CLM 
G4Y03 Chinese Taipei Alignment 8.7 (20.4) 11.2 (17.4) 1.1 (22.0) 
G4Y03 Cyprus Alignment 7.4 (31.9) | —11.2 (43.4) 35.1 (33.9) 
G4Y03 Hong Kong Alignment 13.3 (33.8) | —21.1 (53.6) | —47.9 (37.2) 
G4Y03 Tran Alignment 80.3 (50.3) 19.1 (49.9) 35.9 (95.5) 
G4Y03 Japan Alignment 29.8 (26.1) 29.0 (18.6) 24.5 (22.4) 
G4Y03 Netherlands Alignment | —58.6 (53.1) | —59.1 (63.7) 2.7 (104.1) 
G4Y03 New Zealand Alignment | —32.0 (26.8) 1.5 (20.1) 6.6 (25.3) 
G4Y03 Singapore Alignment 8.9 (36.8) —3.6 (31.8) —10.9 (25.1) 
G4Y03 Slovenia Alignment | —18.5 (24.8) | —22.6 (19.3) | -27.7 (30.7) 
G4Y03 Yemen Alignment | —52.8 (78.2) | —17.5 (28.3) | —55.0 (72.7) 
G4Y03 Canada (Ontario) Alignment 15.6 (29.4) 12.7 (21) 9.1 (28.6) 
G4Y03 Canada (Quebec) Alignment | —24.3 (24.5) | —21.6 (18.2) | —20.1 (25.6) 
G4Y07 Algeria Alignment 84.3 (64.4) 23.7 (45.6) 29.8 (71.9) 
G4Y07 Australia Alignment 31.3 (24.7) 17.7 (24.6) 23.3 (24.2) 
G4Y07 Armenia Alignment —7.6 (46.1) | —11.7 (29) 0.7 (60.7) 
G4Y07 Chinese Taipei Alignment —6.2 (18.8) —4.5 (16) 2.7 (17.6) 
G4Y07 Colombia Alignment 63.0 (39.1) 49.0 (37.0) 44.7 (43.5) 
G4Y07__| Czech Republic -18.7 (35.8) 
G4Y07 Georgia Alignment | —10.6 (57.4) | —64.5 (48.8) 24.7 (92.1) 
G4Y07 Germany Alignment 9.2 (22.1) 6.6 (17.7) 23.5 (24.3) 
G4Y07 Hong Kong Alignment 30.2 (24.5) 54.8 (33.3) 16.5 (22.3) 
G4Y07 Japan Alignment 33.2 (25.9) 27.4 (17.8) 15.1 (29.8) 
G4Y07 Mongolia Alignment 61.5 (47.9) 24.2 (27.4) 30.6 (50.1) 
G4Y07 Netherlands Alignment | —27.3 (79.6) | —25.9 (49.9) | —22.7 (59.4) 
G4Y07 New Zealand Alignment 29.3 (19.3) 20.3 (18.6) 13.3 (19.5) 
G4Y07 Slovenia Alignment 4.7 (14.7) —6.6 (13.2) 9.3 (16.9) 
G4Y07 Sweden Alignment 36.3 (24.7) 32.6 (20.6) 32.3 (35.1) 
G4Y07 Tunisia Alignment | —18.5 (39.9) 31.0 (33) —22.8 (47.7) 
G4Y07 United States Alignment 11.3 (12.4) 19.6 (13.9) 15.3 (12.2) 
G4Y07 Yemen Alignment | —55.1 (70.4) | —58.7 (47.9) | —93.1 (79.0) 
G4Y07 England Alignment —0.3 (32.0) —9.2 (25.5) —5.3 (31.4) 
G4Y07 Scotland Alignment 18.9 (23.2) 9.6 (23.5) 42 (26.4) 
G4Y07 United Arab Emirates Alignment 91.8 (50.8) 71.5 (53.7) 59.6 (76.6) 
(Dubai) 
G4Y07 Canada (Quebec) Alignment 10 (17.7) | —16.3 (20.7) 3.3 (20.1) 
G4Y07 Canada (Alberta) Alignment —0.9 (27.9) | —30.4 (28.7) | —23.2 (24.4) 
G4Y07 Canada (British Alignment 16.1 (25.2) 18.8 (22.4) —0.1 (24.1) 
Columbia) 
G4Y07 United States Alignment —9.9 (35.9) | —52.0 (58.8) | —54.8 (33.7) 
(Minnesota) 
G4yY11 Australia Alignment | —15.5 (16.6) 2.1 (21.6) | 13.4 (18.6) 
G4yY11 Bahrain Alignment 7.9 (14.2) —1.0 (10.3) 1.8 (17.4) 
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Grade/year | Country Predictor SLM MLM CLM 
G4Y11 Botswana Alignment 35.6 (27) 27.4 (19.9) 11.6 (33.6) 
G4Y11 Chile Alignment | —15.5 (34.4) | —23.7 (30.4) —9.1 (30.3) 
G4Y11 Chinese Taipei Alignment —7.7 (8.2) —6.8 (6.5) -7.4 (7.9) 
G4Y11 Croatia Alignment 7.1 (14.4) 0.1 (16.7) 4.7 (19.9) 
G4Y11 Czech Republic Alignment 5.4 (14.4) 7.0 (11.2) —5.8 (16.8) 
G4Y11 Finland Alignment | —11.6 (14.3) | —10.3 (13.1) 3.2 (20.8) 
G4Y11 Honduras Alignment | —46.7 (42.0) —9.6 (32.0) | —35.8 (57.0) 
G4Y11 Hong Kong Alignment —19.0 (46.5) —4.8 (39.3) —16.9 (54.8) 
G4Y11 Tran Alignment 15.6 (23.5) 38.8 (21.7) | -18.4 (34.5) 
G4yY11 Ireland Alignment 3.8 (18.1) 4.7 (15.3) 15.7 (27.6) 
G4Y11 Italy Alignment —7.9 (18.2) 1.5 (20.0) | -12.3 (24.2) 
G4yY11 Korea Alignment | —12.4 (12.7) | —15.1 (9.9) —5.0 (11.3) 
G4Y11 Kuwait Alignment 12.6 (17.1) 15.1 (14.9) 19 (20.2) 
G4yY11 Lithuania Alignment 7.5 (12.1) —5.6 (10.6) 5.7 (14.5) 
G4Y11 Malta Alignment 23.2 (13.9) —5.2 (17.2) 17.4 (26.9) 
G4yY11 Morocco Alignment 49.7 (43.1) 165.2 (88.7) 72.7 (62.3) 
G4Y11 Oman Alignment —1.7 (18.4) 27.5 (15.1) —1.2 (19.6) 
G4Y11 New Zealand Alignment 0.5 (17.0) 6.5 (17.2) 19.6 (20.1) 
G4yY11 Norway Alignment 27.6 (26.0) 31.2 (20.6) 30.9 (24.2) 
G4Y11 Portugal Alignment 1.5 (21.4) 16 (18.1) —3.9 (46.6) 
G4Y11 Romania Alignment | —44.2 (37.8) 7.8 (30.5) | —48.0 (52.6) 
G4Y11 Saudi Arabia Alignment 17.9 (38.8) 36.4 (23.0) 27.0 (40.0) 
G4Y11 Serbia Alignment 16.8 (17.6) 5.7 (19.2) 3.1 (27.6) 
G4Y11 Singapore Alignment 25.1 (25.9) —5.2 (39.1) 15.9 (18.9) 
G4Y11 Slovak Republic Alignment 36.2 (23.2) 3.3 (25.9) 16.9 (32.3) 
G4Y11 Spain Alignment 2.5 (20.6) 20 (22.6) 11.3 (22.5) 
G4yY11 Sweden Alignment 28.8 (22.0) 16.5 (16.4) 29.2 (31.9) 
G4Y11 Thailand Alignment 4.5 (21.3) 7.9 (13.7) —4.8 (24.0) 
G4Y11 United Arab Emirates Alignment 3.7 (19.7) 20.8 (13.5) —6.6 (20.2) 
G4Y11 Tunisia Alignment 14.9 (26.6) 37.4 (21.5) 7.7 (25.7) 
G4yY11 United States Alignment —4.1 (14.7) 14.0 (11.6) | -11.6 (16.4) 
G4Y11 Yemen Alignment 47.5 (62.9) 39.4 (40.5) 18.8 (65.2) 
G4Y11 England Alignment 9.0 (22.3) 31.6 (31.6) 8.4 (30.9) 
G4Y11 United Arab Emirates Alignment 11.2 (31.3) 7.8 (27.9) —7.3 (44.3) 
(Dubai) 
G4Y11 United Arab Emirates Alignment —5.9 (26.5) 0.7 (26.5) 3.8 (28.7) 
(Abu Dhabi) 

G4Y11 Canada (Ontario) Alignment | —23.8 (18.5) | —15.2 (17.2) | -31.7 (23.6) 
G4Y11 Canada (Alberta) Alignment —3.4 (14.2) 20.4 (14.4) —6.6 (15.1) 
G4yY15 Australia Alignment | —35.1 (18.0) | —22.9 (11.9) | —28.7 (18.1) 
G4yY15 Bahrain Alignment 13.1 (12.9) 17.7 (13.8) 29.8 (15.3) 
G4yY15 Bulgaria Alignment 1.5 (39.4) 37.8 (32.8) | —34.7 (46.6) 
G4yY15 Canada Alignment —0.1 (11.0) —-0.3 (9.3) —10.0 (15.2) 
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Table B.14 (continued) 
Grade/year | Country Predictor SLM MLM CLM 
G4yY15 Chile Alignment 3.9 (25.0) —5.6 (20.9) 4.9 (32.8) 
G4yY15 Chinese Taipei Alignment —0.9 (13.4) —0.5 (11.8) 8.1 (12.8) 
G4yY15 Czech Republic Alignment 5.8 (16.2) 9.3 (13.7) 4.7 (17.0) 
G4yY15 Denmark Alignment 16.1 (18.8) 16.9 (13.7) 25.7 (21.3) 
G4Y15 Finland Alignment 1.9 (8.0) —2.3 (9.4) 8.0 (12.9) 
G4yY15 Indonesia Alignment 29.6 (36.3) —3.1 (24.6) 25.1 (37.0) 
G4yY15 Ireland Alignment —9.9 (15.2) —4.6 (10.8) | -17.9 (16.8) 
G4Y15 Italy Alignment —1.5 (17.3) | -14.0 (12.6) | -13.9 (15.3) 
G4Y15 Japan Alignment 0.6 (13.7) 5.5 (10.5) 23.2 (17.0) 
G4Y15 Kazakhstan Alignment 5.3 (31.5) 1.5 (25.9) 1.4 (29.8) 
G4Y15 Korea Alignment 19.7 (14.8) 16.7 (10.2) 18.1 (14.6) 
G4Y15 Kuwait Alignment —3.0 (29.3) | —14.4 (20.8) 3.4 (25.4) 
G4yY15 Morocco Alignment —0.2 (32.7) —6.4 (21.3) 39.3 (34.3) 
G4Y15 Oman Alignment | —15.2 (14.9) | -16.1 (14.1) | —24.5 (18.3) 
G4yY15 New Zealand Alignment | —19.4 (12.1) | —16.8 (10.3) | -19.6 (11.9) 
G4yY15 Norway Alignment 25.1 (13.4) 10.7 (10.8) 8.5 (14.0) 
G4yY15 Serbia Alignment 7.7 (24.5) | —15.3 (18.7) 0.1 (30.1) 
G4Y15 Singapore Alignment | —18.6 (16.4) | —42.1 (24.8) —6.9 (12.5) 
G4yY15 Slovak Republic Alignment —8.0 (20.2) —9.7 (20.1) | —36.9 (25.7) 
G4Y15 Slovenia Alignment 0.8 (14.0) 6.6 (12.1) 2.1 (14.3) 
G4yY15 United Arab Emirates Alignment 10.5 (23.6) | —29.6 (18.0) 2.4 (25.6) 
G4yY15 Turkey Alignment 3.5 (25.7) —1.0 (20.3) 11.9 (23.1) 
G4yY15 England Alignment 4.5 (15.4) 23.8 (26.3) 17.9 (13.8) 
G4Y15 Northern Ireland Alignment —5.9 (19.7) —0.2 (29.0) 0.9 (26.3) 
G4yY15 United Arab Emirates Alignment 9.7 (17.2) | —18.4 (28.5) 19.1 (23.3) 
(Dubai) 
G4yY15 Canada (Ontario) Alignment 0.4 (12.4) | —14.5 (10.6) | -17.6 (13.1) 
G4yY15 Canada (Quebec) Alignment | —34.6 (26.6) | —21.7 (17.2) | —38.4 (24.9) 
G4yY15 Argentina (Buenos Alignment | —18.4 (30.6) | —19.7 (43.8) | —23.2 (33.4) 
Aires) 
G8Y03 Australia Alignment | —45.9 (49.4) | —49.1 (45.3) 33.6 (55.5) 
G8Y03 Bahrain Alignment | —40.6 (23.0) | —20.1 (29.6) | —47.3 (24.4) 
G8Y03 Botswana Alignment | —74.6 (48.4) | —72.5 (53.8) 2.2 (47.0) 
G8Y03 Bulgaria Alignment 19.7 (39.6) 30.1 (27.1) 18.2 (35.4) 
G8Y03 Chile Alignment 39.8 (44.6) | —59.7 (51.1) —5.9 (34.0) 
G8Y03 Chinese Taipei Alignment 12.9 (47.3) 55.9 (66.2) 3.3 (44.1) 
G8Y03 Cyprus Alignment | —15.2 (36.7) | —11.1 (2.1) —2.7 (41.5) 
G8Y03 Estonia Alignment 18.9 (27.9) —4.2 (25.7) | 15.0 (30.8) 
G8Y03 Palestinian Alignment 36.4 (35.8) 25.9 (29.1) 22.3 (37.4) 
G8Y03 Hong Kong Alignment 9.9 (46.2) —6.9 (39.5) —4.7 (34.9) 
G8Y03 Indonesia Alignment | —74.8 (81.4) | —57.5 (54.4) | -65.5 (68.2) 
G8Y03 Tran Alignment 11.7 (32.9) 35.3 (32.7) 1.7 (33.0) 
G8Y03 Israel Alignment 4.6 (35.4) | —73.0 (43.0) | —34.3 (30.6) 
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Table B.14 (continued) 
Grade/year | Country 
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CLM 


G8Y03 Japan 


—89.7 (62.5) 


G8Y03 Lithuania 


2.9 (43.8) 


G8 Y03 Malaysia 


35.7 (27.0) 


G8Y03 Moldova 


24.4 (65.9) 


G8Y03 Norway 


14.5 (28.4) 


G8Y03 Philippines 


29.0 (19.7) 


G8Y03 Romania 


84.1 (70.7) 


G8Y03 Slovenia 


27.8 (26.2) 


G8Y03 Macedonia 


59.4 (42.1) 


G8Y03 Serbia 


—14.1 4.5) 


G8Y07 Algeria 


—2.1 (32.2) 


G8Y07 Australia 


10.5 (18.2) 


G8Y07 Bahrain 


—21.7 (27.9) 


G8Y07 Armenia 


31.1 (24.9) 


G8Y07 Bosnia and Herzegovina 


21.3 (32.7) 


G8Y07 Botswana 


41.6 (38.6) 


G8Y07 Bulgaria 


7.7 (62.6) 


G8Y07 Chinese Taipei 


7.7 (25.0) 


G8Y07 Colombia 


—16.2 (34.3) 


G8Y07 Czech Republic 


—32.9 (44.2) 


G8Y07 Georgia 


3.6 (91.6) 


G8Y07 Palestinian 


—21.6 (22.6) 


G8Y07 Ghana 


9.9 (25.7) 


G8Y07 Hungary 


—31.7 (43.7) 


G8Y07 Tran 


—11.4 (40.1) 


G8Y07 Israel 


—33.5 (46.8) 


G8Y07 Jordan 


—29.3 (26.7) 


G8Y07 Korea 


—21.9 (22.5) 


G8Y07 Kuwait 


31 (52.5) 


G8Y07 Lithuania 


49.3 (51.2) 


G8Y07 Malaysia 


9.6 (25.3) 


G8Y07 Oman 


12 (43.9) 


G8Y07 Norway 


—3.1 (20.1) 


G8Y07 Qatar 


23.9 (31.7) 


G8Y07 Romania 


G8Y07 Saudi Arabia 


G8Y07 Slovenia 


-1.8 (50.9) | 10.9 (35.6) 


15.1 (56.1) 
2.1 (45.6) 
35.5 (36.7) 


G8Y07 Sweden 


—5.0 (18.6) 


G8Y07 Syria 


—26.4 (44.7) 


G8Y07 Tunisia 


1.1 (17.9) 


G8Y07 Turkey 


—8.3 (50.2) 


G8Y07 Egypt 


—37.9 (42.0) 


G8Y07 Serbia 


21.5 (54.9) 


G8Y07 England 


29.9 (29.7) 
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Table B.14 (continued) 
Grade/year | Country Predictor SLM MLM CLM 
G8Y07 | Canada (British Alignment | —28.4 (25.3) | —3.1 (21.6) | -29.9 (23.9) 
Columbia) 
G8Y07 United States Alignment | —62.7 (40.3) | -105.4 (63.8) | —26.0 (39.9) 
(Minnesota) 
G8Y11 Australia Alignment | —54.0 (27.6) -7.0 (29.4) —27.1 (20.6) 
G8Y11 Botswana Alignment —2.7 (19.2) —9.0 (11.4) —6.2 (13.1) 
G8Y11 Chinese Taipei Alignment 60.4 (64.4) —87.6 (64.5) —20.4 (48.2) 
G8Y11 Finland Alignment —3.3 (25.5) 3.0 (32.0) —0.2 (38.7) 
G8Y11 Georgia Alignment 12.6 (33.9) —22.8 (22.7) 28.3 (49.5) 
G8Y11 Palestinian Alignment 48.3 (46.6) —39.1 (29.3) —42.1 (48.7) 
G8Y11 Ghana Alignment 26.9 (24.2) 14.7 (13.6) 36.3 (26.6) 
G8Y11 Honduras Alignment 33.2 (30.9) —36.6 (19.0) 2.7 (33.3) 
G8Y11 Hong Kong Alignment 60.3 (37.9) 45.3 (47.2) | —53.8 (30.7) 
G8Y11 Hungary Alignment 11.5 (38.9) | -10.3 (35.5) | —24.2 (36.6) 
G8Y11 Tran Alignment 5.6 (23.9) 1.8 (22.2) —1.4 (15.6) 
G8Y11 Israel Alignment 40.2 (43.8) —10.2 (37.8) 16.2 (36.5) 
G8Y11 Japan Alignment 7.6 (37.6) 31.3 (48.7) 34.0 (51.6) 
G8Y11 Jordan Alignment 12.5 (34.2) —5.4 (27.4) 41.3 (37.7) 
G8Y11 Korea Alignment 26.2 (21.7) —43.9 (28.6) —35.6 (20.6) 
G8Y11 Lithuania Alignment 25.5 (29.8) -1.9 (35.1) 30.9 (26.5) 
G8Y11 Malaysia Alignment 17.4 (40.7) 14.3 (33.3) —36.0 (34.1) 
G8Y11 Morocco Alignment | —23.3 (25.4) | —25.6 (21.0) | —34.3 (23.0) 
G8Y11 Romania Alignment —16.3 (38.6) 16.8 (37.2) —23.0 (37.8) 
G8Y11 South Africa Alignment 15.4 (12.4) —24.4 (20.9) 7.3 (15.1) 
G8Y11 Sweden Alignment 7.3 (14.1) —9.2 (18.2) 8.0 (21.4) 
G8Y11 Syria Alignment | —27.5 (44.7) —21.6 (27.8) —17.6 (42.0) 
G8Y11 Thailand Alignment 21.3 (27.4) 39.6 (29.2) 2.1 (29.6) 
G8Y11 _| Tunisia 23.5 (13.5) 
G8Y11 Turkey —36.4 (39.3) | —33.4 (30.3) | —30.6 (40.3) 
G8Y11 Macedonia —41.6 (59.0) | —70.0 (76.1) | —46.5 (50.8) 
G8Y11 Canada (Ontario) 4.5 (15.0) —18.1 (19.0) 27.9 (19.8) 
G8Y11 Canada (Quebec) —5.6 (36.5) 97.4 (74.2) —1.3 (30.2) 
G8Y11 Canada (Alberta) 9.2 (21.6) 46.4 (49.6) 21 (24.6) 
G8Y15 Canada —18.6 (16.3) —6.9 (26.3) 1.3 (17.6) 
G8Y15 Chile 11.1 (29.2) 18.0 (19.1) 18.1 (26.7) 
G8Y15 Chinese Taipei 33.5 (26.8) 31.5 (28.8) 46.8 (25.2) 
G8Y15 Georgia —14.8 (27.9) —23.4 (29.2) —6.6 (38.1) 
G8Y15 Hong Kong —1.4 (48.7) | -17.9 (53.0) | —30.9 (51.0) 
G8Y15 Hungary 2.7 (27.7) 12.7 (24.4) 24.4 (25.9) 
G8Y15 Iran 19.9 (36.5) 39.5 (27.8) 37.3 (27.8) 
G8Y15 Ireland —4.1 (17.1) —10.1 (24.3) 7.9 (18.7) 
G8Y15 Italy —9.4 (22.4) 11.0 (26.4) —16.8 (22.9) 
G8Y15 Kazakhstan 82.2 (45.5) —9.5 (35.2) 69.3 (41.7) 
G8Y15 _| Jordan -14.7 (23.7) 
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Grade/year | Country Predictor SLM MLM CLM 

G8Y15 Kuwait Alignment | —32.9 (46.3) | -—34.8 (24.4) | —36.6 (34.0) 
G8Y15 Lebanon Alignment 40.1 (25.8) —1.8 (18.4) 26.8 (22.1) 
G8Y15 Malaysia Alignment 49.6 (36.3) | —133.1 (88.7) —5.8 (27.0) 
G8Y15 Oman Alignment 16.3 (29.4) 25.8 (21.1) 42.6 (26.8) 
G8Y15 New Zealand Alignment 2.0 (27.5) —0.3 (37.1) 3.6 (19.8) 
G8Y15 Slovenia Alignment 20.1 (23.6) 2.9 (22.7) 18.3 (21.2) 
G8Y15 South Africa Alignment | —18.6 (26.0) 3.2 (17.5) —5.0 (20.9) 
G8Y15 United Arab Emirates Alignment | —16.4 (18.3) —5.6 (13.2) 0.3 (17.4) 
G8Y15 Turkey Alignment 32.1 (32.5) 23.6 (22.6) 4.9 (34.5) 
G8Y15 Egypt Alignment 30.9 (62.0) 56.8 (41.7) 40.1 (64.7) 
G8Y15 United States Alignment —2.1 (25.7) | —58.1 (45.0) 27.7 (30.0) 
G8Y15 England Alignment | —43.0 (28.0) | -10.7 (59.8) 4.2 (21.6) 
G8Y15 United Arab Emirates Alignment | —27.0 (56.3) 59.7 (46.6) 39.9 (40.5) 

(Abu Dhabi) 

G8Y15 Canada (Ontario) Alignment | —15.4 (14.1) 6.7 (24.6) —1.8 (18.2) 
G8Y15 Canada (Quebec) Alignment | —66.6 (35.1) | —102.3 (72.3) | —28.8 (27.6) 


Notes Standard errors in parentheses. Grade/year indicates grade level (where G4 and G8 are grade four and 
eight, respectively) and year (where e.g., YO7 is the year 2007), resulting in a composite abbreviation such 
as G4Y07 (which indicates Grade 4, 2007 cycle of TIMSS data). SLM = single level model, MLM = 
multilevel model, CLM = classroom-level model, Alignment = proportion of mathematics topics covered 
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Table B.15 Countries where the relationship of teacher self-efficacy to teach mathematics 
(Prepared) with student achievement was not statistically significant in all three models 


Grade/year | Country Predictor | SLM MLM CLM 
G4Y03 Chinese Taipei Prepared 4.0 (5.0) —1.0 (1.0) 2.3 (5.7) 
G4Y03 Cyprus Prepared 14.8 (10.3) 10.1 (5.9) 18.3 (12.4) 
G4Y03 Hong Kong Prepared 2.1 (6.2) —3.2 (4.0) 4.9 (7.3) 
G4Y03 Tran Prepared 9.8 (10.1) 2.1 (2.0) 20.2 (11.1) 
G4Y03 Japan Prepared 4.2 (5.3) 2.7 (2.1) 4.3 (5.5) 
G4Y03 Netherlands Prepared 3.8 (13.6) | -10.7 (9.8) 2.9 (15.4) 
G4Y03 Singapore Prepared 9.7 (10.5) 1.2 (2.1) 12.6 (8.5) 
G4Y03 Slovenia Prepared 7.6 (8.4) —-0.8 (3.1) 13.1 (9.7) 
G4Y03 Yemen Prepared 21.9 (19.0) | -2.7 (1.7) 24.3 (17.8) 
G4Y03 Canada (Ontario) Prepared 6.2 (7.4) 0.6 (2.9) 8.9 (7.7) 
G4Y07 Austria Prepared 3.9 (2.8) 0.6 (2.2) 3.1 (3.8) 
G4Y07 Armenia Prepared | —2.2 (6.2) 4.4 (7.9) —2.0 (8.5) 
G4Y07 Chinese Taipei Prepared | —1.3 (2.4) —0.2 (1.2) —-1.7 (2.6) 
G4Y07 Denmark Prepared 6.7 (5.4) 2.5 (2.0) 5.2 (5.1) 
G4Y07 Hong Kong Prepared 4.8 (3.8) 0.5 (4.7) 3.4 (3.3) 
G4Y07 Hungary —1.2 (11.0) | -18.5 (31.2) | —0.9 (7.2) 
G4Y07 Japan 4 (2.5) —0.2 (1.9) 6.2 (3.5) 
G4Y07 Mongolia 7.2 (6.7) —2.1 (2.5) 5.4 (6.8) 
G4Y07 Netherlands 15.9 (17.1) | -1.8 (2.2) 9.9 (10.5) 
G4Y07 Singapore 3.9 (6.5) —3.2 (4.2) 2.4 (5.3) 
G4Y07 Sweden 1.8 (4.5) 1.5 (2.4) 0.4 (6.5) 
G4Y07 Yemen 22.0 (11.9) | —-0.3 (3.4) | -10.2 (12.9) 
G4Y07 England 5.6 (6.1) -1.7 8.3) 8.2 (6.0) 
G4Y07 United Arab Emirates 4.7 (11.6) | —0.9 (8.4) —5.2 (20.3) 
(Dubai) 
G4Y07 Canada (Ontario) 10.3 (7.0) 1.2 (2.9) 9.0 (5.9) 
G4Y07 Canada (Quebec) 4.0 (4.9) 1.8 (4.6) 4.0 (6.1) 
G4Y07 Canada (Alberta) 8.9 (5.0) —3.1 (2.9) 7.9 (4.2) 
G4Y07 United States 19.3 (15.0) | —4.1 (3.6) 2.7 (17.3) 
(Massachusetts) 

G4Y11 Australia 6.8 (7.2) 4.8 (7.4) 3.7 (8.4) 
G4Y11 Bahrain 6.7 (7.2) 5.6 (4.9) 4.0 (5.8) 
G4Y11 Chile 7.1 (8.5) 12.1 (10.6) 15.6 (11.9) 
G4Y11 Chinese Taipei 0.2 (3.2) 0.5 (2.2) 2.6 (3.6) 
G4Y11 Croatia 4.5 (5.2) 1.6 (4.5) 5.9 (5.4) 
G4Y11 Czech Republic 2.7 (4.0) —-0.3 (3.1) 0.8 (4.4) 
G4Y11 Honduras 13.2 (9.4) 4 (9.4) —1.7 (12.8) 
G4Y11 Hong Kong Prepared 4.7 (4.8) 3.9 (3.6) —1.5 (4.8) 
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Table B.15 (continued) 
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Grade/year | Country Predictor | SLM MLM CLM 
G4Y11 Hungary Prepared 0.7 (4.7) —3.6 (4.0) 2.7 (5.3) 
G4Y11 Ireland Prepared | —0.1 (5.8) 1.8 (3.8) 0.6 (7.1) 
G4Y11 Italy Prepared 4.9 (4.1) 2.2 (4.3) 6.1 (5.1) 
G4Y11 Japan Prepared 0.7 (2.3) 0.0 (2.4) 2.3 (3.0) 
G4Y11 Korea Prepared 1.1 (2.6) 1.0 (2.2) 1.5 (2.1) 
G4Y11 Kuwait 2.0 (6.5) 5.9 (7.9) 4.8 (8.8) 
G4Y11 Lithuania 5.4 (4.7) 3.2 (3.8) 7A (5.3) 
G4Y11 Malta —2.3 (3.3) —-0.9 (3.7) —2.8 (5.8) 
G4Y11 Morocco 8.5 (6.2) 6.5 (5.2) 2.6 (7.7) 
G4Y11 Netherlands —4.1 (4.8) 1.8 (3.6) —5.3 (4.9) 
G4Y11 New Zealand 8.3 (6.0) 4.4 (3.7) 7.5 (6.2) 
G4Y11 Norway 4.5 (4.8) 2.6 (4.3) 5.4 (4.4) 
G4Y11 Portugal 12.3 (7.8) 4.4 (7.6) 16.2 (14.4) 
G4Y11 Romania 5.9 (7) 4.0 (6.6) 3.6 (10.4) 
G4Y11 Saudi Arabia 9.0 (9.6) 14.4 (8.4) 16 (10.4) 
G4Y11 Serbia 9.9 (5.1) TA (5.8) 11.8 (8.0) 
G4Y11 Singapore 4.3 (6.0) 8.7 (8.6) —0.1 (3.9) 
G4Y11 Slovak Republic 1.4 (4.9) 2.5 (4.9) 2.9 (5.9) 
G4Y11 Spain 2.0 (5.2) 8.0 (4.2) 2.7 (5.7) 
G4Y11 Sweden 2.1 (4.0) 0.0 (2.7) 1.6 (4.5) 
G4Y11 United Arab Emirates —1.8 (4.4) 1.8 (4.4) 2.2 (5.5) 
G4Y11 Tunisia —3.4 (6.9) —-1.9 (4.7) —8.0 (7.1) 
G4Y11 Yemen —12.6 (8.3) —9.8 (6.2) |—-18.2 (10.4) 
G4Y11 Northern Ireland 7.7 (8.5) 9.5 (7.9) 1.5 (8.0) 
G4Y11 United Arab Emirates 5.7 (7.1) 2.1 (9.9) —2.8 (7.7) 
(Dubai) 
G4Y11 United Arab Emirates —2.1 (7.6) 3.6 (6.6) 3.8 (9.6) 
(Abu Dhabi) 

G4Y11 Canada (Ontario) 0.9 (6.3) 0.6 (6.6) 12 (11.6) 
G4Y11 Canada (Quebec) —3.1 (4.3) —0.2 (5.1) 2.2 (5.8) 
G4Y11 Canada (Alberta) 4.3 (6.3) 2.7 (5.2) 5.0 (7.9) 
G4Y15 Australia 2.1 (6.6) 5.2 (3.1) 3.0 (5.6) 
G4Y15 Bulgaria 8.9 (8.0) 13.3 (10.9) 9.0 (9.5) 
G4Y15 Chinese Taipei Prepared —1.9 (3.3) —1.6 (2.9) —3.8 (4.9) 
G4Y15 Cyprus Prepared 12.9 (7.5) 6.1 (5.9) 11.2 (6.8) 
G4Y15 Czech Republic Prepared —2.7 (3.8) —0.4 (3.3) —7.0 (4.1) 
G4Y15 Denmark Prepared —5.7 (9.1) —4.5 (5.3) —5.4 (8.7) 
G4Y15 Finland Prepared 2.6 (2.9) 0.7 (2.9) 5.0 (4.6) 
G4Y15 France Prepared —0.9 (3.9) —0.4 (3.4) —1.2 (5.4) 
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Grade/year | Country Predictor | SLM MLM CLM 
G4Y15 Germany Prepared | —1.1 (3.5) 0.7 (3.1) 0.3 (4.2) 
G4Y15 Hong Kong Prepared 2.5 (3.4) 1.2 (4.5) —1.0 (3.2) 
G4Y15 Hungary Prepared 8.9 (6.1) 4.8 (4.9) 8.7 (6.4) 
G4Y15 Tran Prepared 3.7 (7.2) 8.1 (5.2) —0.6 (10.1) 
G4Y15 Treland Prepared | —3.9 (3.6) 0.7 (2.7) —8.1 (5.0) 
G4Y15 Italy 2.0 (3.7) 5 (4.1) 2.9 (3.7) 
G4Y15 Japan 3.4 (2.4) 3.4 (1.9) 1.0 (3.0) 
G4Y15 Kazakhstan 12.5 (10.2) 10.5 (8.5) 8.3 (10.3) 
G4Y15 Korea 2.8 (2.8) 0.6 (2.1) 1.5 (3.1) 
G4Y15 Kuwait 5.8 (9.3) 9.7 (6.5) 3.2 (8.3) 
G4Y15 Morocco 4.1 (6.6) 1.7 (4.4) 6.5 (7.8) 
G4Y15 Norway 2.8 (4.3) 5.2 (3.3) 0.7 (7.0) 
G4Y15 Poland —4.4 (3.9) —5.1 (3.3) -7.4 (3.9) 
G4Y15 Portugal —11.4 (9.4) —8.0 (8.1) —7.8 (10.6) 
G4Y15 Slovak Republic 5.2 (5.1) 2.2 (3.5) 6.7 (6.0) 
G4Y15 Slovenia —-1.0 (4.1) 0.9 (3.1) 1.4 (4.2) 
G4Y15 Northern Ireland 7.2 (6.5) —9.0 (7.3) 4.2 (7.7) 
G4Y15 Canada (Ontario) 7.3 (5.7) 7.8 (5.9) 1.7 (6.3) 
G4Y15 Canada (Quebec) —10.9 (6.6) | —10.8 (5.9) —4.6 (6.8) 
G4Y15 Argentina (Buenos Aires) —5.5 (11.8) | —2.6 (10.3) 1.2 (9.3) 
G8Y03 Australia -17.1 (25.7) 3.0 (2.8) | —25.0 (19.2) 
G8Y03 Bahrain 1.7 (6.1) 3.9 (2.8) 5.2 (6.9) 
G8Y03 Botswana 1.8 (6.2) —1.6 (2.3) —0.1 (7.2) 
G8Y03 Bulgaria —14.7 (17.8) | -5.3 (9.5) | —21.5 (18.9) 
G8Y03 Cyprus 3.3 (9.7) —7.6 (4.6) —0.1 (9.4) 
G8Y03 Estonia 8.2 (9.2) 2.7 (1.4) 4.9 (8.6) 
G8Y03 Palestinian 18.6 (12.1) 0.2 (2.6) 19.1 (12.0) 
G8Y03 Hong Kong 22.3 (12.1) 2.7 (4.8) 13.8 (7.3) 
G8Y03 Hungary 14.5 (7.5) 1.5 (3.9) 6.1 (7.9) 
G8Y03 Tran 8.1 (7.6) —0.1 (2.8) 4.1 (9.0) 
G8Y03 Israel -7.2 (16.1) 6.3 (5.0) 0.4 (18.2) 
G8Y03 Japan 4.9 (6.9) 0.3 (1.4) 0.3 (6.5) 
G8Y03 Lithuania 2.6 (8.0) —8.3 (5.8) 5.7 (8.5) 
G8Y03 Malaysia 7.7 (11.2) 0.0 (2.3) 12.0 (8.5) 
G8Y03 Moldova -7.9 (9.1) —-7.3 (5.5) —5.8 (9.9) 
G8Y03 New Zealand —25.6 (18.2) 2.2 (3.2) —4.4 (12.3) 
G8Y03 Norway 14.3 (18.9) | —2.6 (1.7) 6.0 (26.4) 
G8 Y03 Philippines Prepared 7.8 (15.9) 4.8 (5.1) —3.6 (12.4) 
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Grade/year | Country Predictor | SLM MLM CLM 
G8Y03 Singapore Prepared 7.5 (9.4) 1.9 (5.6) 0.5 (6.8) 
G8Y03 Slovenia Prepared | —1.6 (8.3) 1.1 (1.7) 2.8 (8.7) 
G8Y03 Macedonia Prepared 4.7 (11.8) 5.8 (3.2) —2.2 (12.2) 
G8Y03 Egypt Prepared 22.5 (15.9) | —4.5 (9.2) 22.8 (16.9) 
G8Y03 Serbia Prepared | —1.1 (6.4) 3.4 (2.5) —2.7 (5.5) 
G8Y03 Canada (Ontario) 6.8 (8.8) 1.0 (2.3) 14.2 (8.9) 
G8Y07 Algeria 3.1 (3.8) 0.7 (1.1) 3.6 (3.7) 
G8Y07 Bahrain 4.3 (4.2) —0.6 (2.8) 4.3 (5.6) 
G8Y07 Armenia 0.6 (4.4) —4.1 (5.9) 2.8 (4.6) 
G8Y07 Bosnia and Herzegovina 0.8 (4.0) 5.4 (4) 0.4 (3.8) 
G8Y07 Botswana 3.8 (7.7) -1.1 @G.4) —1.5 (6.6) 
G8Y07 Cyprus -1L5 (5.1) 3.4 (3.3) —0.7 (4.6) 
G8Y07 Czech Republic —0.8 (14.7) 2.2 (2.3) —6.6 (8.3) 
G8Y07 Georgia —9.9 (12.8) 5.0 (3.8) —2.4 (18.3) 
G8Y07 Palestinian 12.4 (8.9) —3.4 (4.8) 11.4 (9.3) 
G8Y07 Hong Kong 9.0 (7.2) —6.0 (5.7) 0.7 (5.5) 
G8Y07 Hungary —1.6 (6.5) 13.4 (7.3) —9.1 (8.6) 
G8Y07 Indonesia —0.7 (8.6) 7.1 (3.8) —4.0 (8.0) 
G8Y07 Israel 14.4 (11.3) | —2.6 (5.8) 13.6 (8.4) 
G8Y07 Japan 1.9 (4.1) 0.7 (1.2) 5.8 (5.2) 
G8Y07 Korea —0.1 (4.5) 10.2 (7.3) 1.5 (4.4) 
G8Y07 Kuwait —10.5 (8.1) —4.1 (3.5) | -10.6 (8.1) 
G8Y07 Oman 15.4 (9.2) 3.2 (2.3) 15.5 (9.7) 
G8Y07 Norway 8.5 (5.2) —1.8 (1.6) 8.3 (5.8) 
G8Y07 Romania 1.1 (8.9) —0.4 (5.7) —3.1 (9.3) 
G8Y07 Saudi Arabia 3.2 (5.6) -1.7 G.5) —1.0 (6.5) 
G8Y07 Slovenia 1.5 (6.1) —3.0 (3.8) —2.4 (6.6) 
G8Y07 Egypt 15.9 (8.9) 7.7 (3.9) 15.2 (8.7) 
G8Y07 England Prepared 1.7 (16.3) | —7.4 (7.5) —4.2 (17.1) 
G8Y07 Scotland Prepared 6.6 (10.2) | —3.6 (5.6) 2.9 (11.4) 
G8Y07 Spain (Basque) Prepared | —0.2 (6.6) 4.9 (4.2) 8.4 (9.4) 
G8Y07 Canada (British Prepared 16.1 (9.3) 5.7 (3.6) 13.8 (8.5) 
Columbia) 
G8Y07 United States Prepared | —7.0 (19) 4.2 (5.3) —3.2 (13.1) 
(Massachusetts) 
G8Y07 United States Prepared | —8.5 (18.4) 6.1 (9.9) 22.9 (13.9) 
(Minnesota) 
G8Y11 Australia Prepared 4.6 (8.2) 4.7 (8.0) —5.9 (6.6) 
G8Y11 Bahrain Prepared 4.7 (6.4) 11 (5.8) 1.6 (4.4) 
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Table B.15 (continued) 
Grade/year | Country Predictor | SLM MLM CLM 
G8Y11 Chinese Taipei Prepared 0.0 (4.9) —3.3 (9.1) 1.5 (4.2) 
G8Y11 Finland Prepared | —0.7 (2.2) —4.5 (3.1) 0.3 (3.1) 
G8Y11 Georgia Prepared 5.2 (6.4) 8.7 (4.8) 2.8 (8.4) 
G8Y11 Ghana Prepared 4.8 (6.8) 4.9 (6.2) 7.5 (6.9) 
G8Y11 Honduras Prepared 3.4 (7.2) 4.1 (5.5) 3.8 (7.0) 
G8Y11 Indonesia 8.6 (7.8) —1.2 (6.7) 10.6 (7.4) 
G8Y11 Israel 6.0 (8.2) 6.0 (8.4) 6.2 (8.4) 
G8Y11 Italy —0.4 (3.6) 1.0 (3.2) —0.2 (3.3) 
G8Y11 Japan 3.9 (3.6) 6.3 (4.5) 0.1 (3.0) 
G8Y11 Jordan —0.9 (5.5) —1.1 (4.2) -1.7 (6.1) 
G8Y11 Korea 1.3 (3.0) 1.8 (4.4) -0.7 (3.7) 
G8Y11 Lebanon 8.8 (6.1) 4.8 (4.8) 1.3 (5.6) 
G8Y11 Lithuania —3.4 (4.2) —-1.3 (4.6) —2.5 (4.5) 
G8Y11 Malaysia 8.1 (10.3) 10.4 (9.4) —1.7 (8.0) 
G8Y11 Morocco —2.1 (3.7) —0.2 (2.4) —0.8 (3.4) 
G8Y11 Norway —4.0 (3.0) —-1.6 (2.4) —-0.2 (3.1) 
G8Y11 Romania 19 (10.5) 13.7 (12.1) 19.7 (10.9) 
G8Y11 Saudi Arabia 6.5 (12.1) 2.7 (10.0) 9.8 (11.4) 
G8Y11 Singapore —2.7 (4.2) —-1.7 (6.6) —1.1 (2.9) 
G8Y11 Slovenia —0.6 (2.9) —0.8 (3.4) —0.7 (3.0) 
G8Y11 Sweden 0.6 (3.2) 4.6 (3.7) -0.7 (3.7) 
G8Y11 Syria 3.2 (8.5) 0.7 (6.2) 5.0 (8.5) 
G8Y11 Thailand —0.1 (5.6) —1.9 (5.2) 1.9 (5.3) 
G8Y11 Tunisia —2.4 (5.1) —5.1 (3.6) —6.2 (4.3) 
G8Y11 Turkey 4.7 (5.0) 5.9 (3.9) 0.1 (5.0) 
G8Y11 Ukraine 1.0 (4.5) 0.9 (3.5) 2.3 (5.4) 
G8Y11 Macedonia 9.3 (12.1) 10.6 (9.9) 8.9 (8.9) 
G8Y11 United States 9.0 (4.9) —4.8 (13.5) | —0.1 (7.3) 
G8Y11 England 5.7 (16.5) | -16.9 (11.0) 1.5 (9.2) 
G8Y11 United Arab Emirates —2.9 (6.2) —0.8 (9.8) 3.3 (6.7) 
(Dubai) 
G8Y11 United Arab Emirates —1.8 (7.9) —2.6 (8.3) 0.7 (8.5) 
(Abu Dhabi) 
G8Y11 Canada (Ontario) Prepared 0.2 (3.4) —2.1 (5.4) —2.2 (4.3) 
G8Y11 Canada (Quebec) Prepared 0.5 (4.9) 12.4 (12.0) 1.5 (4.4) 
G8Y11 Canada (Alberta) Prepared 3.7 (8.5) 1.1 (14.6) 3.8 (8.4) 
G8Y15 Australia Prepared 4.0 (6.2) 5 (1.1) 4.0 (5.8) 
G8Y15 Chile Prepared | —5.5 (10.4) | —4.4 (8.0) 0.8 (8.8) 
G8Y15 Chinese Taipei Prepared 1.2 (3.5) —1.6 (6.5) —4.9 (6.6) 
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Table B.15 (continued) 


Grade/year | Country Predictor | SLM MLM CLM 
G8Y15 Georgia Prepared -7.7 (6.5) —5.5 (5.4) | —-11.0 (10.0) 
G8Y15 Hong Kong Prepared —2.6 (4.3) —9.3 (7.8) —8.2 (5.2) 
G8Y15 Hungary Prepared | —0.1 (5.3) —2.0 (5.7) —1.2 (5.9) 
G8Y15 Tran Prepared | —1.9 (7.7) 0.3 (5.8) —-7.5 (6.5) 
G8Y15 Ireland Prepared 5.1 (4.9) 1.6 (3.9) —-1.6 (3.4) 
G8Y15 Israel Prepared | ~—1.8 (7.0) —4.8 (5.2) —8.5 (5.9) 
G8Y15 Italy Prepared 3.1 (4.2) 5.2 (3.9) 1.2 (4.4) 
G8Y15 Kazakhstan Prepared | —4.0 (10.1) | —3.5 (7.3) —1.6 (9.6) 
G8Y15 Jordan Prepared 5.4 (5.1) 5.4 (2.8) 3.1 (4.5) 
G8Y15 Korea Prepared 1.9 (3.1) 3.5 (4.6) —1.0 (3.2) 
G8Y15 Lebanon Prepared 4.8 (7.2) 0.4 (3.0) —2.2 (6.1) 
G8Y15 Malaysia Prepared | —4.4 (8.0) 0.1 (17.9) | —4.6 (5.2) 
G8Y15 Oman Prepared 1.5 (6.5) 4.0 (4.4) 0.4 (6.3) 
G8Y15 Norway Prepared 3.4 (5.2) 5.0 (5.1) —0.5 (6.3) 
G8Y15 Qatar Prepared 0.2 (8.6) 7.3 (9.0) 1.8 (7.4) 
G8Y15 Turkey Prepared 1.1 (4.9) 1.2 (3.6) —3.8 (5.8) 
G8Y15 Egypt Prepared 3.0 (8.8) 7.1 (3.9) 6.9 (8.7) 
G8Y15 United States Prepared -1.9 (4.1) —5.5 (6.9) —4.5 (4.6) 
G8Y15 Canada (Ontario) Prepared | —2.3 (7.2) 0.3 (7.3) —3.0 (6.0) 
G8Y15 Canada (Quebec) Prepared | —6.9 (4.5) | -12.5 (8.5) —5.6 (3.6) 


Notes Standard errors in parentheses. Grade/year indicates grade level (where G4 and G8 are grade 
four and eight, respectively) and year (where e.g., YO7 is the year 2007), resulting in a composite 
abbreviation such as G4Y07 (which indicates Grade 4, 2007 cycle of TIMSS data). SLM = single 
level model, MLM = multilevel model, CLM = classroom-level model, Prepared = index (1-4) of 
self-efficacy to teach mathematics 


